Elevated Latency on Query Execution (Retool Cloud)
minor- Started
- Oct 5, 11:49 AM EDT
- Resolved
- Oct 12, 6:42 PM EDT
- Duration
- 174h 53m
Retool's updates (11)
- investigating · Oct 5, 11:49 AM EDT
Some customers may experience slower than normal query load times. We're actively investigating and will provide updates as we learn more.
- investigating · Oct 5, 6:06 PM EDT
Some users may see improved query latency, but we are continuing to investigate.
- monitoring · Oct 5, 10:44 PM EDT
Some changes have been deployed, and we are monitoring the results
- investigating · Oct 6, 10:40 AM EDT
We are seeing intermittent spikes in query latency, as we saw earlier. A temporary mitigation is in place, and we're continuing to investigate the root cause. We'll share an update as soon as we have more information.
- investigating · Oct 6, 3:03 PM EDT
The temporary mitigation we applied earlier is still in place, and some users have reported significantly improved query latency. We're still investigating the root cause and will share another update as we learn more.
- monitoring · Oct 6, 7:47 PM EDT
A fix has been implemented and we are monitoring the results.
- investigating · Oct 7, 11:21 AM EDT
This morning we again saw high query latency. We were able to mitigate it temporarily, and our investigation into the root cause is ongoing. We'll share another update as we learn more.
- investigating · Oct 7, 6:19 PM EDT
Additional changes have been deployed as part of a more permanent fix, alongside the temporary fix already in place. We'll monitor through tomorrow morning's peak traffic to confirm their impact and will share another update then.
- monitoring · Oct 8, 10:23 AM EDT
A fix has been implemented and we are monitoring the results.
- monitoring · Oct 8, 5:20 PM EDT
We have not seen the same query latency issues since the fix was applied. We’re continuing to monitor closely and will share another update if anything changes.
- resolved · Oct 12, 6:42 PM EDT
Query performance is back to normal and the incident is resolved.
What our checks recorded
We checked Retool 576 times from Retool's start time to its resolution, once every five minutes. Every one of them reached Retool's site, so this incident did not show up as a failed page load. Our check loads the public site; it cannot see an API error or a broken feature. Median response time was 130 ms, against 147 ms over the 24 hours before. Retool posted the incident at Oct 5, 11:49 AM EDT.
Updates are Retool's own words from its status page (status: resolved); we never add an inferred cause. The vendor's incident page ↗