Degraded OTEL ingestion in US region
minor- Started
- Sep 2, 12:02 PM EDT
- Resolved
- Sep 2, 12:56 PM EDT
- Duration
- 54 min
Langfuse's updates (4)
- investigating · Sep 2, 12:02 PM EDT
We are observing OTEL ingestion backlog in US region
- identified · Sep 2, 12:24 PM EDT
A deployment rollout coincided with a traffic spike, resulting in an unusually large backlog. Adjusting our autoscaling to absorb the backlog faster.
- monitoring · Sep 2, 12:49 PM EDT
Backlog is almost completely gone. Ingestion delay is < 1 min.
- resolved · Sep 2, 12:56 PM EDT
Completely back to normal
What our checks recorded
We checked Langfuse 11 times from Langfuse's start time to its resolution, once every five minutes. Every one of them reached Langfuse's site, so this incident did not show up as a failed page load. Our check loads the public site; it cannot see an API error or a broken feature. Median response time was 139 ms, against 200 ms over the 24 hours before. Langfuse posted the incident at Sep 2, 12:02 PM EDT.
Updates are Langfuse's own words from its status page (status: resolved); we never add an inferred cause.