Data ingest delays impacting small percentage of datasets

Incident Report for Honeycomb

Resolved

We have completed rebalancing and monitoring for stability.
Posted Nov 10, 2022 - 00:19 PST

Monitoring

Backlog is now caught up; we are still rebalancing workloads and will monitor for stability after rebalancing is completed.
Posted Nov 09, 2022 - 23:42 PST

Update

We are correcting the workload imbalance; query delay has recovered but estimated time to recovery for SLO evaluation for 5% of traffic is <20 minutes.
Posted Nov 09, 2022 - 23:32 PST

Identified

We are investigating delays in queryability of recently ingested data for a <10% subset of traffic.
Posted Nov 09, 2022 - 23:20 PST
This incident affected: ui.honeycomb.io - US1 Querying and ui.honeycomb.io - US1 Trigger & SLO Alerting.