Incident Resolved: CTIM-921
No further reports of problems from National Pen since the fix was deployed earlier today. We will follow up regarding the affected order remediation process. Incident is contained.
Please contact CTIM or see ticket for more details.
UPDATE:
National Pen’s Tech Teams have fixed the root cause and new orders should be OK.
Impacted orders have been bulk-cancelled which should remove them from the queue, but attempts to resend them in bulk have failed. As such Agents are going to have to remediate them manually.
UPDATE:
National Pen teams have found the issue and are working on a fix. This fix will only help with new orders, currently impacted orders will need to be rejected and resubmitted. The team is currently discussing the best approach to handling the remediation of existing orders.
UPDATE:
Teams are online and continue to investigate, no major updates at this time.
UPDATE:
Escalation sent to: CT Incident Management
National Pen tech team is online and looking into the issue. Redirecting escalation from Artwork Generation to CTIM as we dont have National Pen tech team in Pager Duty to page.
New Incident: CTIM-921
Priority: Critical
Escalation sent to: Artwork Tech: Artwork Generation for review.
No previews are being generated