All systems and backlogged queues recovered, shortly prior to midnight central time. This incident is now fully resolved.
Posted Mar 07, 2023 - 08:54 CST
Update
Engineers are continuing to monitor recovery and increase queued processing. Delays of automations processing, segment.com events such as (clicks, opens, site visits) remain delayed and are expected to continue processing over the next several hours. Campaign sending, triggered by these types of events may also be delayed. Resolution of this incident will be declared when queued messages have completed processing.
Posted Mar 06, 2023 - 18:10 CST
Update
Engineers continue to monitor recovery. Automations processing delays and segment.com integration event processing remains delayed.
Posted Mar 06, 2023 - 17:10 CST
Update
Engineers continue to monitor and apply performance improvements to expedite recovery. At this time, delays in processing and intermittent errors may continue to be experienced until full resolution is declared.
Posted Mar 06, 2023 - 14:21 CST
Update
Engineers continue to monitor recovery. At this time, delays in processing and intermittent errors may continue to be experienced until full resolution is declared.
Posted Mar 06, 2023 - 13:07 CST
Monitoring
A fix has been implemented, however delays in automations & integrations processing may continue as queues begin to process.
Posted Mar 06, 2023 - 12:08 CST
Update
Engineers continue to investigate and have identified additional impact to Campaign pages, list pages are intermittently failing to load. Engineers have identified mitigation steps, which are underway.
Posted Mar 06, 2023 - 11:16 CST
Identified
Engineers are actively engaged in investigating a performance degradation issue beginning at 8:20 am CST Customers may experience delays in automations process and segment.com integration event processing.
Posted Mar 06, 2023 - 10:38 CST
This incident affected: US Region (Automations, Campaigns, E-commerce, Integrations, Platform Availability) and Site (Tracking).