Inventory and Delivery Planning
Checkout - Spike in use of Fallback Speeds in shipping options due to EDO Timeout
Incident Resolved: CTIM-969
We are containing this incident at this stage while the Onboarding and Curation team continues their investigation to determine the root cause. We will await their updates and confirmation on the root cause and action items.
Please contact CTIM or see ticket for more details.
UPDATE:
The Onboarding and Curation team suspects that a specific database query caused a significant slowdown in one of their database instances. They plan to optimize the process of retrieving and validating the related data to prevent this issue in the future. The next update will be provided once a mitigation is identified.
UPDATE:
Dispatch squad noticed a response time spike from a downstream service and identified the issue in the Product Group Service and requested the Onboarding and Curation team to investigate, providing the affected timeframe. Net update will be shared when we hear something from them.
UPDATE:
Escalation sent to: PCD: Dispatch Squad
This escalation is being redirected to the Dispatch squad who will investigate it during their business hours. CTIM will provide updates once their investigation begins.
New Incident: CTIM-969
Priority: High
Escalation sent to: PCD: Product Operations for review.
Between 10:30 PM and 10:45 PM UTC on 22 November, there was a significant increase in the use of Fallback Speeds for shipping options caused by an EDO Timeout. Although fallback speeds ensured users could access shipping options, this workaround is suboptimal, particularly during the busy holiday season. Further investigation is needed to address and prevent this issue. We request the team to prioritize investigating this issue during business hours.