Resolved -
This incident has been resolved.
May 11, 09:37 PDT
Update -
Activity Log recovered at 07:00 UTC-07:00 today (about 6.5h ago). All streams are caught up.
May 8, 13:46 PDT
Update -
All activity log streams are now caught up, except for the query runs table, which should have all data since the start of the incident backfilled by 1600 PDT.
May 8, 03:40 PDT
Update -
We are continuing to monitor for any further issues.
May 8, 03:22 PDT
Update -
We believe that the activity log should now be caught up.
May 8, 03:21 PDT
Monitoring -
Backfilling of the past 2h30m of data is now in progress and should complete shortly.
May 8, 03:13 PDT
Identified -
Activity log delay is rising, and is more than an hour behind due to a MySQL replica failover. Estimated recovery by 04:00 PDT.
May 8, 01:00 PDT