SLO processing delays

Incident Report for Honeycomb

Resolved

SLOs are functional again, and data has been backfilled for the period of the incident.
Posted Sep 30, 2021 - 16:51 PDT

Update

SLOs are functional again, and data has been backfilled for the period of the incident.
Posted Sep 30, 2021 - 16:50 PDT

Update

Recovery continues, a small portion of teams continue to have delayed SLO reporting, and we are working to recover that data.
Posted Sep 30, 2021 - 16:33 PDT

Update

Recovery continues, a small portion of teams continue to have delayed SLO reporting.
Posted Sep 30, 2021 - 15:46 PDT

Update

Recovery continues, some teams continue to have delayed SLO reporting.
Posted Sep 30, 2021 - 15:12 PDT

Update

Recovery continues, some teams continue to have delayed SLO reporting.
Posted Sep 30, 2021 - 14:42 PDT

Update

Recovery continues, some teams continue to have delayed SLO reporting.
Posted Sep 30, 2021 - 14:09 PDT

Monitoring

We have identified the issue and have implemented a fix which we are continuing to monitor. SLOs continue to be delayed at this time
Posted Sep 30, 2021 - 13:40 PDT

Identified

Our SLO batch processing system appears to be having issues. SLO data is getting delayed and alerting may be similarly be delayed. Triggers and other query types are functional.
Posted Sep 30, 2021 - 13:09 PDT
This incident affected: ui.honeycomb.io - US1 Querying.