WebSSO & ConfirmID Services Unavailable
Incident Report for Imprivata
Postmortem

On Thursday 6/23/2022, at 5:20am ET, monitoring alerted the Imprivata team to a potential issue with a service causing degradation in Confirm ID and WebSSO functionalities.

Upon investigation, it was determined that the cause of the issue was unusually high CPU utilization by one of the cloud services. Emergency maintenance was then performed, although these steps did not fully resolve the issue. At 9:20am, the team completed a rollback to the previous stable version. Tests confirmed that functionality was fully restored, and the system was healthy.

At 9:50am, the incident was declared as resolved and the team moved to standard monitoring procedures.

Mitigation and Action Item(s)

  • On-going investigation to isolate the specific change that caused the higher-than-normal CPU and to deploy a fix
  • Process adjustments and additional testing to enhance load test runs prior to changes in production.
Posted Jun 29, 2022 - 11:44 EDT

Resolved
Our systems checks are complete and our cloud platform monitoring shows that this issue has been resolved.
Should you continue to experience issues with WebSSO or ConfirmID please contact Imprivata Customer Support for further assistance.
Posted Jun 23, 2022 - 09:40 EDT
Update
- Platform fixes now show service levels returning to normal.
- Our DevOps & QA teams continue to test and monitor platform performance.
- We expect all customer services should now be functioning normally again.
Posted Jun 23, 2022 - 09:25 EDT
Monitoring
- Platform fixes are now being implemented and we are monitoring the results.
- Our DevOps and QA teams are running further tests to confirm service levels.
Posted Jun 23, 2022 - 09:20 EDT
Identified
- Our DevOps team are now preparing platform fixes for the affected components, which we expect to be implemented by 0945EDT.
- Imprivata recognises the significant impact of this outage and are working as swiftly as possible to restore service.
- Manual OTP Verification is a potential workaround where Imprivata ID push notifications are not being received.
Posted Jun 23, 2022 - 08:50 EDT
Investigating
Our DevOps teams are engaged and investigating the cause of this issue.
Posted Jun 23, 2022 - 08:36 EDT
This incident affected: Imprivata OneSign (Web SSO SAML Identity Provider - User Authentication, Web SSO SAML Identity Provider - Administrative Services) and Imprivata Confirm ID (Remote Access – Cloud-Based, Mobile EPCS – Authentication, Clinical Workflows – Authentication with Imprivata ID, Imprivata ID token).