GPU Node Provisioning Failure
none- Started
- Sep 16, 11:00 AM EDT
- Resolved
- Sep 30, 7:16 PM EDT
- Duration
- 344h 16m
WellSaid Labs's updates (3)
- identified · Sep 16, 11:00 AM EDT
We are currently experiencing issues provisioning new GPU nodes in our GKE environments. This is causing a reduced capacity in the number of requests we're able to serve simultaneously, leading to longer response times and clips failing to generate.
- monitoring · Sep 17, 12:00 AM EDT
Provisioning of new GPU nodes in our GKE environments began returning to normal overnight. We are now seeing only minimal delays in node creation, and service performance has largely stabilized. We are working on expanding our infrastructure and are actively monitoring node availability.
- resolved · Sep 30, 7:16 PM EDT
This incident has been resolved.
Updates are WellSaid Labs's own words from its status page (status: resolved); we never add an inferred cause. The vendor's incident page ↗