Cloud provider outage. Users unable to login
critical- Started
- Oct 20, 2:51 AM EDT
- Resolved
- Oct 20, 8:40 AM EDT
- Duration
- 5h 49m
Encord's updates (5)
- investigating · Oct 20, 2:51 AM EDT
Due to cloud provider outage, users are unable to login and use the platform. We are working with the cloud provider to resolve it as soon as possible.
- identified · Oct 20, 3:51 AM EDT
Since approximately 06:00 UTC (07:00 BST) today, customers have been unable to log in to or use the Encord platform. The cause is an ongoing incident at our cloud infrastructure provider, Google Cloud, in the europe-west2 (London) region. Google has confirmed high packet loss on internal networking (incident MRCWYZJ). Google has also confirmed issues affecting multiple services in the region (incident KMRBWCZ), including managed databases and Kubernetes. As a result, our application servers cannot reliably reach our database and supporting services. Our engineering team is working with Google Cloud and is restoring the affected components where we can. Google has not yet given an estimated resolution time. We are sorry for the disruption. We’ll post our next update as soon as we can.
- identified · Oct 20, 5:03 AM EDT
Google Cloud has now confirmed the issue affecting its europe-west2 (London) region on its public status page. You can follow Google’s latest updates here: • Incident details: <https://status.cloud.google.com/incidents/Xs2XTHzAVC9YnafnPyzj> • Europe regional status: <https://status.cloud.google.com/regional/europe#status-table> We’re continuing to work on restoring service and will post our next update as soon as we can.
- monitoring · Oct 20, 8:39 AM EDT
EMEA service is fully restored. No expected impacts on data integrity.
- resolved · Oct 20, 8:40 AM EDT
EMEA service is fully restored. No expected impacts on data integrity.
What our checks recorded
We checked Encord 70 times from Encord's start time to its resolution, once every five minutes. Every one of them reached Encord's site, so this incident did not show up as a failed page load. Our check loads the public site; it cannot see an API error or a broken feature. Median response time was 196 ms, against 214 ms over the 24 hours before. Encord posted the incident at Oct 20, 2:51 AM EDT.
Updates are Encord's own words from its status page (status: resolved); we never add an inferred cause.