Artwork Processing

Issue with Sherbert

Resolvido

Incident Resolved: CTIM-990
The change has been deployed, and HEIF files can now upload successfully from iOS devices. Teams continue to monitor the health of the system, but everything appears to be running normally. This issue is now considered contained, and no remediation is needed for previous failures.
Please contact CTIM or see ticket for more details.

Atualizado

UPDATE:

The Sherbert API Team has investigated two different approaches to resolving this problem. The first would be to upgrade the underlying library used for HEIF, the second would be to convert the HEIF file directly to a JPG using a different third party package only in these failing scenarios. Given the risks involved, it was deemed that option 2 is the best short-term solution as it isolates the changes to just when these errors occur. The team is planning to deploy this solution before Cyber Monday, and CTIM will update this incident again once deployment has taken place.

Atualizado

UPDATE:

This issue appears to be a common and widespread issue related to HEIC files generated by iOS 18 devices. There is no general solution available, but a library update from 1.11.0 to 1.18.0 may help with the issue, the team is assessing the risk of deploying this updated library and will determine whether it introduces more risk than benefit. The next update will be as soon as we have more information or within 24 hours. Whichever comes first.

Atualizado

UPDATE:

Teams have concluded that there is something related to the files and GraphicsMill/ImageMagick causing the issue, and have requested additional resources be paged to assist with further investigation.

Atualizado

UPDATE:

All teams are now online and investigating. Current troubleshooting seems to point to an uncalibrated colorspace in the the MacOS preview > Inspector - this should fall back to Uploads V1 when this happens, but isn’t - the investigation continues.

Atualizado

UPDATE:

AT: Infrastructure Squad is online and investigating:

“Upon first glance, the content-type for the shared .heic file looks fine.
I was able to reproduce the failed image processing using this same file POST’d using MacOS & Postman.
This leads me to believe the issue is file related, which could explain the intermittent behavior across a variety of OS and Browser combinations”

investigation continues to identify root cause.

Investigando

New Incident: CTIM-990
Priority: Critical
Escalation sent to: Artwork Tech: MOS & DocExt for review.
please treat this as a major production incident with Sherbert. HEIC files have been supported for a long time, so something must be wrong if they’re giving a processing error.