Incident Resolved: CTIM-1939
Production has confirmed no additional issues with scanning batches today. Incident is contained.
Please contact CTIM or see ticket for more details.
UPDATE:
Support team remediated the stuck batch that was reported once again. Awaiting confirmation from the fulfiller before containing this incident.
UPDATE:
Once issue was reported and the team is currently online to check on this further. Next update in max 1 hour
UPDATE:
Escalation sent to: Paperless: D’Ninjas
Requesting team to check on the reported issue once again
UPDATE:
Paperless teams have been working to remediate the batches in Casino workflows. We will confirm with production before containing the incident.
UPDATE:
Batching in silkscreen and sublimation workflows is now functioning as expected. We will keep the incident open for 1 hour in case additional problems are reported.
UPDATE:
Paperless teams are still investigating the issue. The batching problems are likely caused by ongoing degradation of an upstream service (GWM). Once that is restored, Paperless teams can work on remediation and sync issues. Next update in 30 minutes.
UPDATE:
Ongoing issues in AWS are likely to continue affecting batching processes. Support teams are aware and working the issue. Next update in 30 minutes.
UPDATE:
We’re keeping this channel open specifically for the ongoing issue in NP MEX. Support teams have indicated that they’ve scaled the infrastructure to handle the large influx of requests after the AWS outage earlier today. Next update in 30 minutes or sooner.
UPDATE:
Support teams have confirmed this is related to an ongoing incident (CTIM-1937). We will provide additional updates and information in that channel.
New Incident: CTIM-1939
Plant: NatlPen-MEX
Priority: Critical
Escalation sent to: Paperless: Rhydon Squad for review.
National Pen (MEX) is reporting that batches in the silkscreen process (Acetate Paper Ready) are currently stuck, preventing production on multiple machines in that workflow.