Incident Resolved: CTIM-1729
Customers confirmed that they are able to process orders now. Incident is contained.
Please contact CTIM or see ticket for more details.
UPDATE:
The team has fixed the root cause and orders are now being processed, though at a slower pace. The order count will be shared shortly. We will monitor until all affected orders are cleared.
UPDATE:
Transmissions for the API Push adapter are currently not being processed. The issue is still unresolved and team continues to investigate this with high priority. We will share an update as soon as we have more information or max in 30 minutes.
UPDATE:
After creating DB index, team confirms the improvements but still monitoring the flow. Next update when we get more information on the improvements or max in 30 minutes.
UPDATE:
Team has created the DB index and is monitoring its impact on the affected flow. Additionally, orders stuck since 6:00 AM UTC have been re-transmitted using a script. Next update in max 30 minutes.
UPDATE:
Merchant orders are currently stuck in ‘In Progress’ status under Transmission Details. Appropriate Teams are already online and investigating this issue with high priority. Next update will be shared when we get more information from the teams or max in 30 minutes.
Adding all potentially affected Business Units into this escalation.
UPDATE:
Escalation sent to: PCD: Trdelnik Squad
Adding another Support team for assistance. Next update in max 30 minutes
UPDATE:
Team is online and actively looking into the issue with high priority. Next update in max 30 minutes.
New Incident: CTIM-1729
Priority: Critical
Escalation sent to: PCD: Product Operations for review.
Approximately 539 Printdeal merchant orders are currently stuck in ‘In Progress’ status under Transmission Details. Requesting team to look into this immediately