Imprivata Cortext incident

Incident Report for Imprivata

Postmortem

On September 15, 2021, at 4:00am EDT, the Cortext team upgraded the production servers to tune database and directory performance. Our initial monitoring of the metrics during the first few hours after the upgrade showed expected improvements.
At 10:45am, EDT, the team observed a sudden database load increase. This was followed by gradual performance degradation impacting login and messaging delivery for all customers by 11:15am EDT. Our initial mitigation steps, including a rolling restart of servers, did not improve the situation. The team then used a contingency plan to roll back the morning’s update. The team determined that the message delivery and login was fully restored around 12:45 EDT. The outage lasted approximately 90 minutes.

Posted Sep 21, 2021 - 17:09 EDT

Resolved

The issue with Cortext logins and messaging has been resolved. We thank you for your patience. A brief incident summary will be posted here in 1-2 business days.

Posted Sep 15, 2021 - 13:01 EDT

Update

Imprivata continues work on the issue with Cortext. Both logins and messaging may be affected. Another update will be sent within 30 minutes.

Posted Sep 15, 2021 - 12:34 EDT

Update

Imprivata is continuing work to resolve the ongoing Cortext issues. Logins and messaging are impacted. Another update will be sent soon.

Posted Sep 15, 2021 - 12:03 EDT

Investigating

Imprivata has been made aware of an ongoing Cortext production incident. Currently logins and sending messages may be affected on all clients. Ongoing steps are being done now to mitigate the issue. We will provide another update in 30 minutes.

Posted Sep 15, 2021 - 11:34 EDT

This incident affected: Imprivata Cortext (Messaging, Third Party SMS).