Update - The deploy has completed and we are seeing logs flowing to both the dashboard UI as well as to log streams. We will continue to monitor logs this afternoon to ensure there were no adverse effects, and to validate that these changes improve the log loss and delayed logs some users were seeing.
Jun 16, 17:13 UTC
Jun 16, 17:13 UTC
Update - We believe we have identified all the causes behind log loss and log delays that some users are experiencing, and we pushed a fix this morning. Unfortunately, the fix inadvertently dropped logs from log streams from 13:46-14:17 UTC. We reverted to investigate.
We have fixed the issue and are redeploying. We will be monitoring to ensure there is no disruption to log forwarding. We will update the status when the deploy completes.
Jun 16, 16:44 UTC
We have fixed the issue and are redeploying. We will be monitoring to ensure there is no disruption to log forwarding. We will update the status when the deploy completes.
Jun 16, 16:44 UTC
Monitoring - We have reverted the changes that caused the disruption. Services should be receiving logs via the dashboard and log streams. We are working to make sure the next rollout goes more smoothly and will update this status when the changes are deployed.
Jun 13, 22:29 UTC
Jun 13, 22:29 UTC
Investigating - During a rollout of a change to improve logging reliability some of the necessary components failed to update properly. This caused logs and log streams for some services to be interrupted. We are working on resolving this as quickly as possible.
Jun 13, 19:46 UTC
Jun 13, 19:46 UTC
Monitoring - We've deployed a change to one of our clusters that our initial testing has shown improves log reliability. We will be monitoring our logging system to ensure there's no negative impact on any user logs. If the change performs well in production we will then roll it out to all of our clusters.
Jun 13, 17:15 UTC
Jun 13, 17:15 UTC
Identified - Since 05/18 a small number of users have been experiencing sporadic logging issues that we determined are more systemic than initially thought. During our investigation we made several changes to the existing infrastructure that did not alleviate the problem and ultimately made the decision to implement a more robust tool to better handle our log forwarding needs. The changes necessary to implement this tool are in place and will be deployed shortly. Once deployed, we will monitor the results and ensure that the new system performs as we are expecting.
Jun 9, 23:31 UTC
Jun 9, 23:31 UTC
Render Dashboard
?
Operational
Render Website
?
Operational
Render API
?
Operational
Static Sites
?
Operational
Oregon
Operational
Web Services
?
Operational
Cron Jobs
?
Operational
Background Workers
?
Operational
Builds and Deploys
?
Operational
PostgreSQL
Operational
Redis
Operational
Web Services - Free Tier
Operational
Frankfurt
Operational
Web Services
?
Operational
Cron Jobs
?
Operational
Background Workers
?
Operational
Builds and Deploys
?
Operational
PostgreSQL
Operational
Redis
Operational
Web Services - Free Tier
Operational
Ohio
Operational
Web Services
?
Operational
Cron Jobs
?
Operational
Background Workers
?
Operational
Builds and Deploys
?
Operational
PostgreSQL
Operational
Redis
Operational
Web Services - Free Tier
Operational
Singapore
Operational
Web Services
?
Operational
Cron Jobs
?
Operational
Background Workers
?
Operational
Builds and Deploys
?
Operational
PostgreSQL
Operational
Redis
Operational
Web Services - Free Tier
Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.