Resolved - Incident Report Performance Issues
Service Disruption 2019-05-23

Customer Business Impact:
From approximately 2019-05-23 16:15:00 UTC until 2019-05-23 16:37:00 UTC all MindTouch sites experienced a significant service disruption.

Problem Summary:
An unidentified error when entering parameters into our release script caused servers to be terminated unexpectedly.

Root Cause:
MindTouch weekly release process begins with the initiation of a pre-release script. This script prompts us for data that will be deployed to our production environments. While running the script, an incorrect input was accepted by the script as a valid release date/branch name.

As a result, release_20190516 application servers were not available to handle service requests and service was interrupted.

Corrective Actions:
The root cause in this instance was an original error type that had not been encountered previously. We have scheduled an update to the build and deployment scripts that will scan and identify errors of this nature going forward prior to build initiation.
May 24, 16:35 UTC