SLO processing has caught up and we are not seeing anymore lag.
Posted Feb 24, 2022 - 12:34 PST
Monitoring
Our scaling and workload distribution efforts were effective and we're now seeing signs of recovery. SLO processing is catching up. We will update to resolved when the processing of SLO data has caught up completely.
Posted Feb 24, 2022 - 09:18 PST
Identified
We have found what we believe to be the key elements behind the processing lag, and are in the process of scaling and better spreading the workload; we're starting to see progress and catching up take place, but we're still lagging behind on SLO data. All other products (and triggers) remain unaffected.
Posted Feb 24, 2022 - 08:39 PST
Investigating
We have detected delays on our processing of SLO data. Delay is partial and some data may be multiple minutes behind. We are still trying to identify a specific trigger while mitigating the impact.
Posted Feb 24, 2022 - 07:27 PST
This incident affected: ui.honeycomb.io - US1 Trigger & SLO Alerting.