Resolved
After ongoing monitoring and remediation of the root cause, we have confirmed the underlying issue is resolved.
Monitoring
The API in the EU region has been fully operational since around ~2:40 PM UTC. We are continuing to monitor it closely.
Monitoring
A temporary fix has been put in place. EU-based projects should now be back to normal.
Identified
We've identified a set of specific requests which are impacting the stability of the containers. We are working on a temporary block for these requests while we work on a longer term fix to handle them more gracefully.
Investigating
Upgrading this incident from Degraded service to Partial Outage as we've seen increased sustained error rates while we continue to work on resolving the issue.
Investigating
At around 2:30 AM GMT, the API containers associated with EU projects became unhealthy. This has led to increased latency and failure rates for API requests.