We have not observed any further errors and all previously impacted sites are fully operational.
Equinix has put in place a fix and shared that the outage was caused by an erroneous prefix list update based on an incomplete prefix list returned by RADb (https://www.radb.net/) during an outage.
Render will continue to explore redundancy options to mitigate the failure mode that occurred here in the future.
Customers who implemented a temporary A record change per the mitigation suggested earlier in the incident must revert that change as soon as possible and point back to the old IP.
Posted May 10, 2021 - 15:24 UTC
Update
We are continuing to monitor for any further issues.
Posted May 10, 2021 - 09:11 UTC
Monitoring
We are observing that traffic has recovered to pre-incident levels, and most sites are now back up. We are continuing to monitor and awaiting confirmed resolution from Equinix.
Posted May 10, 2021 - 08:46 UTC
Update
We are starting to see some recovery in the impacted network layer and are continuing to monitor.
Posted May 10, 2021 - 08:37 UTC
Update
We are still awaiting resolution from Equinix, and continuing to pursue alternate solutions. We will post further updates once there are new developments.
Customers can continue to take advantage of the temporary mitigation we shared in our prior update.
Posted May 10, 2021 - 08:13 UTC
Update
Equinix has informed us that resolution of the underlying issue may be delayed, so we are continuing to aggressively pursue alternate mitigations.
We continue to encourage customers to consider the temporary migration posted in our earlier status update in order to bring back connectivity to their sites.
Posted May 10, 2021 - 06:56 UTC
Update
Our provider, Equinix, is still investigating the networking issue. In parallel, we are exploring alternate mitigations that we can apply until the underlying issue is resolved.
Impacted customers can still apply the temporary mitigation mentioned in an earlier update in order to restore connectivity to their sites. If applying this mitigation, the configuration changes will need to be reverted following the resolution of the incident.