Artwork Processing
Maestro is experiencing issues due to the massive influx once the COAM issue
The technical teams have completed their root cause and corrective action analysis. The post-mortem can be view here: https://cimpress-support.atlassian.net/wiki/spaces/DCR/pages/15741091912/2023-10-17+Maestro+is+experiencing+issues+due+to+the+massive+influx+once+the+COAM+issue
Workflows is stable now. We have requested users to retry previous backlog in batches.
It has been an hour since this incident was contained, and we have not seen a recurrence. We are marking this incident as resolved. Within five business days, we will post a link the the post mortem of this incident here.
Traffic is now being directed back to eu-west-1 as per usual.
It appears that the backlog is beginning to diminish.
The technical team is still working on this issue. We will update this status page soon.
The team is actively enhancing the infrastructure to support new executions, preventing overburdening the existing infrastructure.
Services in the US-east-1 have stabilized. While processing times in eu-west-1 have improved, they remain significantly high, averaging around 3 minutes. The team is actively engaged in rerouting traffic for new executions, shifting them from eu-west-1 to us-east-1, with the aim of easing the backlog’s burden.
Maestro is experiencing issues due to the massive influx once the COAM issue is resolved. We have scaled to our max and are working through the backlogs. Engineers are now looking to temporarily increase our max replicas to work through the backlog faster and address 504s