Prepress

Rendered docs with black lines on product

Resolved

Incident Resolved: CTIM-1853
Support Teams have contained or resolved the issue. Please contact MIS PBM or see ticket for more details.

Updated

UPDATE:

Currently waiting on confirmation of the data needed for remediation. We will update again once we can verify the customer care teams are able to move forward with the process.

Updated

UPDATE:

Support team has shared the list with Windsor team. Next update after we gear back from support team.

Updated

UPDATE:

Due to the significant amount of manual effort required to generate an accurate impact list, it’s not expected to be completed until Monday. We will follow up with the dev team during their next business hour to confirm the ETA.

Updated

UPDATE:

Latest update from the support team is that progress is being made on the new list but will require another couple hours due to the manual work involved. They expect to be near completion by 4pm ET and we will provide another update at that time.

Updated

UPDATE:

New version of the impact list is still in-progress (for items to be proactively reprinted). Support teams are currently evaluating the output of items that have been caught in production and placed on-hold (then re-rendered) as the problem seems to persist for some items.

Next update in 2 hours as the various support teams work through the ongoing remediation process.

Updated

UPDATE:

Re-rendering has been confirmed to fix the impacted items. Next steps are to reprint items currently on hold and proactively re-render any affected items once identified. Team is working on preparing a list of impacted items (VP Item IDs) within the next hour so plants can take action. Next update in 1 hour.

Updated

UPDATE:

The team continues to work on the script to get the list of impacted items. Team has asked for an extended time of 2 more hours to complete it. Next update in 2 hours from now on.

Updated

UPDATE:

The team started working on the script during the early EST morning hours and provided an ETA of 2 hours. CTIM will follow up in 2 hours and provide an update here.

Updated

UPDATE:

As per the latest update, both parties have agreed to reconnect today around 5:30 PM IST to review the list again. CTIM will follow accordingly.

Updated

UPDATE:

The list is in the final review stage and will be shared once completed. This will then require remediation efforts with help from Vista CARE teams which we’ll begin tomorrow AM EST. Next update will come once we have the list and are ready to engage additional teams for reorders.

Updated

UPDATE:

Artwork teams continue to actively work the issue. The impacted items list is being narrowed down through a custom script which is still processing. We will send another update by 6pm ET or sooner once the script has completed.

Updated

UPDATE:

Impacted order list generation is still in-progress by Artwork teams. They have indicated it will take a few more hours to complete. We will check in and update again at 4pm ET (1.5 hours).

Updated

UPDATE:

Some of the impacted orders may still be in the CDN and will require another 8 hours to refresh. The next update will be shared at 5 PM EST.

Updated

UPDATE:

Remediation of the shared impacted orders has been completed. The team is now checking if there are any remaining affected orders that still require remediation. Next update in max 1 hour

Updated

UPDATE:

Artwork team is still working on remediating the affected items at the highest priority, which is expected to take a few more hours. CTIM will follow up in 2 hours and share the findings.

Updated

UPDATE:

Artwork teams are currently working on the remediation effort of the affected items at highest priority. Next update in max 3 hours.

Updated

UPDATE:

We’re currently awaiting completion of the regenerated assets from Artwork teams. There will be some additional time needed to filter the impact lists by business unit and send those items to the appropriate teams for processing. CTIM will follow up with the dev squads tomorrow AM (IST, ~1-2 hours) to begin remediation of the affected orders.

Updated

UPDATE:

A list of affected items has been created and Artwork teams are working to regenerate the assets for those orders. Once that process is complete, we will need to filter the list for Vista-specific items to be remediated. Next update in 1 hour.

Updated

UPDATE:

Since the issue had a wide impact, additional time will be needed to generate the list of affected orders. We will continue to provide updates and share the list asap.

Updated

UPDATE:

This issue is related to a previous incident that has had a fix applied so that new orders should not be affected. Previous order remediation is still in-progress. We will wait to contain this instance of the problem until we confirm the next steps for order remediation.

Updated

UPDATE:

Prepress teams are currently reviewing the issue. Next update in 30 minutes.

Investigating

New Incident: CTIM-1853
Plant: WND
Priority: Critical
Escalation sent to: Artwork Tech: Support for review.
Production in WND has reported black lines on rendered docs in multiple workflows. The symptoms are similar to the ongoing issue previously reported by Pixart. Artwork teams have already been working the issue.