SaaS US Outage

Incident Report for Incode

Resolved

This incident has been resolved; we see normal traffic within SaaS environment.
Data Synchronization: We are currently performing data synchronization from the secondary environment back to the primary environment. This process requires a minimum of 72 hours to complete. During this time, all services remain fully operational.
Posted Oct 21, 2025 - 02:58 UTC

Update

AWS services have stabilized, and we have successfully switched traffic back to our primary US SaaS environment. All services are now operating normally from the primary region.

Data Synchronization: We are currently performing data synchronization from the secondary environment back to the primary environment. This process requires a minimum of 72 hours to complete. During this time, all services remain fully operational.

We will continue to monitor service performance closely and provide updates on the completion of data synchronization.
Posted Oct 21, 2025 - 02:54 UTC

Update

AWS services have stabilized, and we have successfully switched traffic back to our primary US SaaS environment. All services are now operating normally from the primary region.

Data Synchronization: We are currently performing data synchronization from the secondary environment back to the primary environment. This process requires a minimum of 72 hours to complete. During this time, all services remain fully operational.

We will continue to monitor service performance closely and provide updates on the completion of data synchronization.
Posted Oct 21, 2025 - 02:10 UTC

Update

AWS services have stabilized, and we have successfully switched traffic back to our primary US SaaS environment. All services are now operating normally from the primary region.

Data Synchronization: We are currently performing data synchronization from the secondary environment back to the primary environment. This process requires a minimum of 72 hours to complete. During this time, all services remain fully operational.

We will continue to monitor service performance closely and provide updates on the completion of data synchronization.
Posted Oct 21, 2025 - 01:38 UTC

Update

AWS services have stabilized, and we have successfully switched traffic back to our primary US SaaS environment. All services are now operating normally from the primary region.

Data Synchronization: We are currently performing data synchronization from the secondary environment back to the primary environment. This process requires a minimum of 72 hours to complete. During this time, all services remain fully operational.

We will continue to monitor service performance closely and provide updates on the completion of data synchronization.
Posted Oct 21, 2025 - 01:07 UTC

Update

AWS services have stabilized, and we have successfully switched traffic back to our primary US SaaS environment. All services are now operating normally from the primary region.

Data Synchronization: We are currently performing data synchronization from the secondary environment back to the primary environment. This process requires a minimum of 72 hours to complete. During this time, all services remain fully operational.

We will continue to monitor service performance closely and provide updates on the completion of data synchronization.
Posted Oct 21, 2025 - 00:33 UTC

Update

AWS services have stabilized, and we have successfully switched traffic back to our primary US SaaS environment. All services are now operating normally from the primary region.

Data Synchronization: We are currently performing data synchronization from the secondary environment back to the primary environment. This process requires a minimum of 72 hours to complete. During this time, all services remain fully operational.

We will continue to monitor service performance closely and provide updates on the completion of data synchronization.
Posted Oct 21, 2025 - 00:02 UTC

Update

AWS services have stabilized, and we have successfully switched traffic back to our primary US SaaS environment. All services are now operating normally from the primary region.

Data Synchronization: We are currently performing data synchronization from the secondary environment back to the primary environment. This process requires a minimum of 72 hours to complete. During this time, all services remain fully operational.

We will continue to monitor service performance closely and provide updates on the completion of data synchronization.
Posted Oct 20, 2025 - 23:33 UTC

Update

AWS services have stabilized, and we have successfully switched traffic back to our primary US SaaS environment. All services are now operating normally from the primary region.

Data Synchronization: We are currently performing data synchronization from the secondary environment back to the primary environment. This process requires a minimum of 72 hours to complete. During this time, all services remain fully operational.

We will continue to monitor service performance closely and provide updates on the completion of data synchronization.
Posted Oct 20, 2025 - 23:03 UTC

Update

AWS services have stabilized and we have successfully switched traffic back to our primary US SaaS environment. All services are now operating normally from the primary region.

Data Synchronization: We are currently performing data synchronization from the secondary environment back to the primary environment. This process requires a minimum of 72 hours to complete. During this time, all services remain fully operational.

We will continue to monitor service performance closely and provide updates on the completion of data synchronization.
Posted Oct 20, 2025 - 22:34 UTC

Update

AWS services have stabilized and we have successfully switched traffic back to our primary US SaaS environment. All services are now operating normally from the primary region.

Data Synchronization: We are currently performing data synchronization from the secondary environment back to the primary environment. This process requires a minimum of 24 hours to complete. During this time, all services remain fully operational.

We will continue to monitor service performance closely and provide updates on the completion of data synchronization.
Posted Oct 20, 2025 - 22:33 UTC

Update

We continue to operate in our secondary environment (Disaster Recovery Plan active) with the following known limitations: Face Authentication and SSO Authentication in Incode Dashboard.
Posted Oct 20, 2025 - 21:54 UTC

Update

We continue to operate in our secondary environment (Disaster Recovery Plan active) with the following known limitations: Face Authentication and SSO Authentication in Incode Dashboard.
Posted Oct 20, 2025 - 21:20 UTC

Update

We continue to operate in our secondary environment (Disaster Recovery Plan active) with the following known limitations: Face Authentication and SSO Authentication in Incode Dashboard.
Posted Oct 20, 2025 - 20:51 UTC

Update

We continue to operate in our secondary environment (Disaster Recovery Plan active) with the following known limitations: Face Authentication and SSO Authentication in Incode Dashboard.
Posted Oct 20, 2025 - 20:21 UTC

Update

Following the recent fix, we are observing considerable improvement in latency across our Secondary SaaS US environment. Response times have returned to expected levels.

We continue to operate in our secondary environment (Disaster Recovery Plan active) with the following known limitations: Face Authentication and SSO Authentication in Incode Dashboard.
Posted Oct 20, 2025 - 19:36 UTC

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Oct 20, 2025 - 18:52 UTC

Investigating

We are currently experiencing a Major Outage affecting our US SaaS environment. Our team is actively investigating the issue and working to restore service as quickly as possible.

We will provide updates as more information becomes available.
Posted Oct 20, 2025 - 18:22 UTC

Update

Update - We are still currently experiencing degraded performance with multiple services including id-processing and SSO. Our team is actively troubleshooting the issue.
Posted Oct 20, 2025 - 18:02 UTC

Update

We are still currently experiencing degraded performance with multiple services including id-processing. Our team is actively troubleshooting the issue.
Posted Oct 20, 2025 - 17:40 UTC

Monitoring

Traffic switched to secondary environment due to AWS degradation. Service restored to normal performance. Face Authentication with temporarily limitations.
Posted Oct 20, 2025 - 16:50 UTC

Update

To maintain service availability and improve response times while AWS resolves the infrastructure issues, we are activating our Disaster Recovery Plan (DRP) and switching traffic to our secondary environment. This is a standard procedure designed to minimize impact to your operations during infrastructure incidents.
Posted Oct 20, 2025 - 16:30 UTC

Update

We continue experiencing elevated response times may affect time-sensitive operations in our SaaS US environment due to AWS service degradation. Our team is actively implementing mitigations to minimize impact while AWS works toward resolution.
Posted Oct 20, 2025 - 15:23 UTC

Identified

Incode internal team has determined that the degradation and increased response latency is related to ongoing AWS services degradation. We are closely monitoring the situation and AWS's progress toward full stability.
Posted Oct 20, 2025 - 14:51 UTC

Investigating

We have identified and are actively investigating a service degradation affecting our US SaaS environment. This issue is characterized by increased response latency that may impact your operations.
Posted Oct 20, 2025 - 14:31 UTC
This incident affected: Incode Omni (Omni US).