Pricing
Ecommerce seeing over 70 failures each in Workflow price and Declared value calculator
Incident Resolved: CTIM-733
Optic Squad confirmed that the affected workflows have been reduced and everything is stable. Incident is contained.
Please contact CTIM or see ticket for more details.
Incident Updated: The stuck orders have been retried, and their pricing has been updated. Can’t determine the exact reason but it appeared to have no impact on the majority of the orders they were handling or a significant portion of them. Observed more than 400 workflows impacted at the highest point and now recovering. We will monitor for some time and then contain the incident.
New Incident: CTIM-733
Priority: Critical
Escalation sent to: PCD: AT Squad for review.
Ecommerce encountering more than 70 failures in both the Workflow price and Declared value calculator and they assume that it is due to the same underlying service issue. Based on initial observations it appears that they are experiencing intermittent failures. After the backoff period is over their system automatically retries workflows that have failed. They noticed that the pricer service is experiencing issues and showing 5xx errors.