Artwork Processing
All businesses using Sherbert are impacted.
Incident Resolved: CTIM-852
A plan has been made to mitigate any service downtime for the next 18 hours through manual intervention until the mongoDb index is fully established. We are containing the incident again but will keep an eye on the channel in case of problems.
Please contact CTIM or see ticket for more details.
UPDATE:
Teams have been working to better understand the issue and how to prevent future occurrences. We will keep following the progress and update once we’re sure it won’t reoccur.
UPDATE:
Support teams have indicated that this incident is ongoing as there have been continued problems throughout the day. We will continue to send updates until we can confirm the issue has been fully resolved.
Incident Resolved: CTIM-852
As per the latest update, the Artwork team informed us that there was a backlog in job processing, which caused a scale-up. As the backlog was processed, the infrastructure scaled down, and connection availability increased, allowing processes that could not connect to the database to connect successfully again. Since the issue is resolved, as confirmed by the Artwork team, we are moving the case status to ‘contained.
Please contact CTIM or see ticket for more details.
UPDATE:
Currently, the Artwork team has joined the Slack channel and is investigating the issue.
New Incident: CTIM-852
Priority: Critical
Escalation sent to: Artwork Tech: Support for review.
All businesses using Sherbert are completely down as far as Studio is aware; they can’t upload anything or access any previous uploads.