MindTouch degraded performance: Slow load time
Incident Report for MindTouch
Postmortem

Customer Business Impact:
On 20231222 between 15:35 - 17:23 UTC all sites experienced increased latency and 50x errors.

Details:
Our systems were subjected to short burst of http requests from distributed IP addresses. The nature of the requests and their coordinated nature suggests that this was a Denial of Service attack.

The supposed attack was designed to specifically circumvent the response times of the AWS Web Application Firewall.

Once the origin and pattern of attack had been identified we were able to implement countermeasures and stabilize service.

Actions:
Our operation team has deployed additional resources and has identified a series of measures that will reduce exposure to this attack pattern going forward.

Posted Dec 28, 2023 - 22:12 UTC

Resolved
This incident has been resolved.
Posted Dec 23, 2023 - 02:08 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Dec 22, 2023 - 19:55 UTC
Investigating
We are currently investigating this issue.
Posted Dec 22, 2023 - 16:09 UTC
This incident affected: Application (General Service), Search, In-Product Contextual Help, MindTouch Success Center, Analytics, and Geoblocking for Russia.