Degraded performance due to spike of requests

Incident Report for retraced

Resolved

This incident has been resolved.
Posted Feb 08, 2024 - 14:19 CET

Update

We have mitigated the root cause and worked on preventions in our publicly available API. All systems have been instantiated into fully operational state.
Posted Feb 08, 2024 - 14:19 CET

Update

We are continuing to monitor for any further issues.
Posted Feb 08, 2024 - 14:17 CET

Update

As mitigation measure, we disabled our public api which was the main trigger of the query. This ensures us that all other services are running as expected.
Posted Feb 08, 2024 - 12:11 CET

Monitoring

We have identified and through identification immediately applied a fix. We are observing the system now in its performance. It was a single query which has supposedly led to a sudden unexpected occupation of the database related to product variation updates. The engineering team is working on a prevention plan now.
Posted Feb 08, 2024 - 11:30 CET

Investigating

We have a spike in requests and that may make the platform appear being down. This is related to just very slow responses and we are investigating.
Posted Feb 08, 2024 - 11:26 CET
This incident affected: Platform (dashboard.retraced.com).