Incident Resolved: CTIM-1864
Full remediation plan underway and being actioned by National Pen Mexico. Incident Resolved.
Please contact CTIM or see ticket for more details.
UPDATE:
Escalation sent to: Paperless: Symphony Squad
The revised impacted list has been shared by the Support team. We are now engaging another Support team for the next actions. Next update will be provided once we have more details, or within 2 hours.
UPDATE:
Support team is pulling data to verify the list of orders impacted by the file version issue. Once this list is available, we will check with the Fulfiller Oracle team for further remediation actions. Next update will be shared once we have the list from Support or in a maximum of 2 hours and CTIM will follow up with the next team accordingly.
UPDATE:
Dev team and fulfillers are currently reviewing and revising the impacted order list for further checks. The next update will be provided once we have more details, or within 2 hours at the latest.
UPDATE:
National Pen has gathered additional items that need to be reviewed for rejection/reinjection. We will share this with the dev teams for investigation during their next business hour.
Next update once we have more details from the Paperless teams on the updated list.
UPDATE:
CT and NP teams have corrected and reinjected all orders that had not been already produced or shipped. We are confirming the completed (produced/shipped) items with the NP team before reinjecting those into production as well.
Incident Resolved: CTIM-1864
Teams continue to work on remediation efforts while incident remains Contained. Next update when remediation is completed.
Please contact CTIM or see ticket for more details.
UPDATE:
Oracle team has shared there is no process in which they can generate at their level a new MRD for these products. Prepress team has identified 9 items that they will have to remediate manually.
UPDATE:
Support team is awaiting for Fulfiller team’s confirmation to proceed with rejection and reinjection of items. Next update we we get a response from them.
UPDATE:
A list of a few impacted orders has been shared. Support teams will reject and reinject the items to confirm the resolution. The next update will be provided once we have more information on the progress or within 2 hours at the latest.
UPDATE:
Escalation sent to: Paperless: Symphony Squad
Adding another support team to check with rejection and reinjection of impacted orders.
UPDATE:
CTIM will follow up with the Paperless teams during their business hours (IST). Next update once we’re able to test one of the problematic orders and determine what the process should be to fix the rest of the affected items.
UPDATE:
Escalation sent to: Artwork Tech: Prepress
Adding Prepress teams for assistance.
UPDATE:
Artwork teams are currently reviewing the issue. Next update in 30min.
UPDATE:
Escalation sent to: Artwork Tech: Support
Production in National Pen (MEX) is experiencing a reoccurrence of an issue where .ai file versions are not recognized in Pad Print workflows. This is currently halting all production in that area. Requesting assistance from Artwork teams to review prepress parameters.
UPDATE:
Production states that the ‘File Version’ error persists and has halted work for the whole workflow. Currently re-engaging Paperless dev squad for assistance.
Incident Resolved: CTIM-1864
Paperless teams have determined issue appears to be from internal system issue and have requested National Pen IT investigate. Next update when there is a response from National Pen.
Please contact CTIM or see ticket for more details.
UPDATE:
National Pen has completed test, awaiting development feedback. Next update max 1 hour.
UPDATE:
Support Team has shared the files with National Pen team to check which file has issue, awaiting response from National pen team. Next update after we hear back from NP team,
UPDATE:
The team is still investigating the NatlPen-MEX PadPrint issue as a high-priority incident. Next update in 1 hour or less.
New Incident: CTIM-1864
Plant: NatlPen-MEX
Priority: High
Escalation sent to: Paperless: D’Ninjas for review.
CTIM is currently investigating, with high priority, an incident affecting PadPrint files at NatlPen-MEX. All machines have been stopped for the past two hours, causing production loss. Next update in 30 minutes or less.