Elevated error rate - AWS CDN (Singapore)
resolved- Started
- Aug 7, 11:14 AM EDT
- Resolved
- Aug 7, 7:14 PM EDT
- Duration
- 8h
Hugging Face's updates (2)
- update · Aug 7, 11:14 AM EDT
Some file downloads from our Singapore region (ap-southeast-1) failed with 500 errors during two short periods, starting at 15:14 UTC. Downloads are normal again now. The cause is not fully resolved, so short bursts of errors can return until our fix is in place. Other CDN regions are not affected. We are sorry for the disruption.
- resolved · Aug 7, 7:14 PM EDT
Our fix is now in place and this incident is resolved. Applying it caused one final short period of failed downloads between 23:00 and 23:14 UTC, as we noted could happen. Downloads from the Singapore region have been stable since, and we have added capacity to prevent this from recurring. We are sorry for the disruption.
What our checks recorded
We checked Hugging Face 96 times from Hugging Face's start time to its resolution, once every five minutes. Every one of them reached Hugging Face's site, so this incident did not show up as a failed page load. Our check loads the public site; it cannot see an API error or a broken feature. Median response time was 131 ms, against 153 ms over the 24 hours before. Hugging Face posted the incident at Aug 7, 11:14 AM EDT.
Updates are Hugging Face's own words from its status page (status: resolved); we never add an inferred cause.