We have confirmed that services have been restored with the upstream provider, and have confirmed restoration of our own services as well. Deploys and scheduling workloads have recovered. We will continue to monitor platform recovery.
Posted Feb 14, 2024 - 20:27 UTC
Update
We have implemented mitigations for most users, but are still experiencing delays scheduling new user workloads due to an outage by an upstream provider. We are continuing to investigate ways we can mitigate.
Posted Feb 14, 2024 - 19:22 UTC
Identified
We're still seeing continued degradation, and we're working on further mitigations.
Posted Feb 14, 2024 - 18:49 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Feb 14, 2024 - 17:38 UTC
Investigating
We are currently investigating an issue running builds for some users in the Oregon region.
Posted Feb 14, 2024 - 17:19 UTC
This incident affected: Oregon (Web Services, Cron Jobs, Builds and Deploys, PostgreSQL, Redis).