We believe the system is currently back under stable trigger performance.
Posted Aug 01, 2022 - 13:24 PDT
Monitoring
We have managed to re-stabilize the trigger runs. We've also identified a few factors that could be in play to explain what has been going on and will try to confirm them more solidly before resolving the incident. Performance is still not entirely as good as usual, but is currently within acceptable ranges.
Posted Aug 01, 2022 - 11:57 PDT
Identified
The issue hasn't been transient and has started happening again. We are looking deeper in underlying and contributing factors to re-establish quality of service.
Posted Aug 01, 2022 - 11:00 PDT
Monitoring
We have identified hosts that were having bad performance and have replaced them. We will keep an eye to make sure everything was transient.
Posted Aug 01, 2022 - 09:40 PDT
Investigating
We’re currently investigating issues and possible delays in evaluating triggers.
Posted Aug 01, 2022 - 09:12 PDT
This incident affected: ui.honeycomb.io - US1 Trigger & SLO Alerting.