Incident Resolved: CTIM-833
All affected services have returned to normal functionality. Incident is contained.
Please contact CTIM or see ticket for more details.
UPDATE:
PASTA and PI services have returned to normal performance levels. There remains a slightly higher-than-average error rate in MAISE though it’s still trending downward.
UPDATE:
Problematic node was terminated and pods recycled. Error rate metrics have normalized (by ~15:33 ET). Will monitor before containing the issue.
UPDATE:
Updating status page category to Product Config.
New Incident: CTIM-833
Priority: Critical
Escalation sent to: PCD: Strandbeest for review.
Strandbeest reported slowness and errors related to the PASTA and Product Intermediary (Product Catalog) services that are used by Vista. Initial investigation shows some issues with the Mongo db connections used by the affected services. More details in channel.