Incident Resolved: CTIM-1315
Changes have been deployed to production. Orders that were submitted during the incident are being indexed and will be available once that is completed. Incident is now contained.
Please contact CTIM or see ticket for more details.
UPDATE:
The deployment on the testing environment was successful and new orders can now be searched. The Optics squad is now deploying the changes to the Production environment. Incident will be moved to contained status once the deployment is completed. The next update will be provided within a maximum of 1 hour.
UPDATE:
The Optics squad is currently deploying a fix in the testing environment. If it works as expected, they will proceed with the deployment in the production environment which may take about an hour. Once everything is completed and stable, they will reindex the missed items to ensure visibility in the respective applications. The next update will be provided when we hear back from the Optics squad on a successful deployment or within a maximum of 2 hours.
UPDATE:
As per the latest update, the Dev squad has initiated a deployment that will stop the indexing of new orders which is required for the search functionality. Once that is complete, they will update the configuration and redeploy the change to ensure everything works smoothly. ETA has not been provided but CTIM will continue following up on this and will share an update when we hear from them or maximum in 2 hours.
UPDATE:
The Optics squad is still working on the configuration mapping and we have not received any updates from them so far. CTIM will follow up on this again after 1 hour. The next update will be shared when we hear from Optics squad or maximum in 1 hour.
UPDATE:
Impact- Only New Orders created in OOps & Complaints created after 09.30am IST / 04.00am UTC won’t be available in Search.
UPDATE:
The Optics team has decided to focus on the mapping configuration involved in the search index as the escalation raised with the AWS team has not helped resolve the issue. This is expected to be completed within the next 2 hours including a buffer period. The next update will be provided in 2 hours.
UPDATE:
eCommerce informed us that this issue was caused by an indexing-related problem in Elasticsearch, which has already been escalated to the AWS team with high priority. Next update after we hear back from AWS team.
New Incident: CTIM-1315
Priority: High
Escalation sent to: eCommerce Support for review.
Currently Order Search in OOps, Complaint Apps and all other applications which consumes the search functionality provided by order processing team from eCommerce.
This has been caused due to indexing related issue at Elasticsearch which is already escalated to AWS team with high priority.
Note-No Order Flow is impacted due to this issue and all would be processed without any issues. This issue Order Item search will only be observed after Order is injected into Order Operations or any Complaint is created.