Incident Resolved: CTIM-1870
Support Teams have contained or resolved the issue. Please contact MIS PBM or see ticket for more details.
UPDATE:
A prod patch to fix this issue has been released. We will provide another update once all plants have deployed the patch.
UPDATE:
The support team will check if they can gain access to a proper test environment. If that isn’t possible, they will proceed with releasing the patch to production. Next update by end of day today.
UPDATE:
Fulfillment dev teams were delayed in validating the fix for this issue due to infrastructure problems that affected the test environment. We will follow up tomorrow and provide an update in the morning (ET).
UPDATE:
Teams continue to investigate now that we are within their business hours. Next update by EOD (ET timezone)
UPDATE:
Team is not currently within business hours. Next update will be when team has come online and confirmed they are looking into this issue again.
UPDATE:
The Dev team is working on a fix to prevent future issues. Additional details on their progress will be shared once available. The next update is expected when their business hours begin ~3–4 hours from now.
UPDATE:
The dev squads have identified the issue with the how the notification email address is selected. They are planning to patch this query logic to mitigate future problems, though there remain some rare situations where even the updated query cannot accurately determine the correct email. Additional input from other Fulfillment teams needed to evaluate this fix.
There’s also some inconsistencies with how the orders were placed that require clarification from the merchant.
We will follow the ongoing investigation and provide additional updates tomorrow.
UPDATE:
The supporting dev squads have identified the problem. It has to do with the logic used to pull the email for a customer based on physical address. They are currently evaluating possible fixes to the issue. We will provide another update by EOD on the progress.
UPDATE:
Escalation sent to: Fulfillment: Planning/Data Investigation
Requesting Fulfillment Planning/Data Investigation to investigate during business hours.
UPDATE:
Escalation sent to: Fulfillment: Logistics: Gengar
Adding Gengar team for further assistance on this issue.
New Incident: CTIM-1870
Plant: VEN
Priority: High
Escalation sent to: Fulfillment: Logistics Support for review.
CTIM is currently investigating, at high priority, an incident affecting Venlo where the shipping confirmation for order VP_X94XD152 was sent to a previous customer’s email address, raising data privacy concerns. Next update in 1 hour or less.