Header Image
All Systems Operational

Updated a few seconds ago

Website




Operational

Broadcast




Operational

Dashboard




Operational

Call Widget




Operational

Agent Tool Desktop




Operational

Agent Tool Mobile App




Operational

Liveshopping Mobile App




Operational

Liveshopping Player




Operational

REST API




Operational

Multistreaming




Operational

Chat Widget




Operational

Chat Mobile App




Operational

Chat Dashboard




Operational

0

Active Incidents

2

Incidents Last 30 Days

0

Maintenances Last 30 Days

External Services

Amazon Web Services

Google Cloud

Slack

Status.io

History (Last 30 days)

December 3, 2024 – API DowntimeOperational

Incident Status

Operational


Components

Broadcast, Dashboard, Call Widget, Agent Tool Desktop, Agent Tool Mobile App, Liveshopping Mobile App, Liveshopping Player, REST API


Locations

Global, Frontend US, Frontend EU, Backend US, Backend EU



December 4, 2024 2:32PM UTC
[Resolved] On December 3, 2024, starting at approximately 13:42 UTC, a configuration issue in our backend APIs caused temporary disruptions to services like the Agent Tool and the ability to start pre-recorded broadcasts. We promptly identified the root cause and swiftly restored full functionality across all impacted regions. Root Cause: The incident was traced to a misconfiguration of the API's egress settings. An update earlier in the day changed the egress configuration, which advertently blocked outgoing HTTP requests to critical services, disrupting functionality across several endpoints. Key Timestamps: - 13:42 UTC: First recorded timeout in API logs. - 15:28 UTC: Egress settings were corrected in the staging environment, and tests confirmed restored functionality. - 15:37 UTC: Changes deployed to production resolved the issue for all affected services. - 15:41 UTC: Agent Tool and other impacted services confirmed operational. Future Prevention Measures: - Egress Configuration Monitoring: Implement monitoring for egress configuration changes to detect and alert on potentially detrimental modifications. - Automated Testing: Expand automated tests in staging to include checks for egress functionality before deploying to production. - Ping Monitoring Enhancements: Review and enhance existing production monitoring to include outgoing traffic verification. The incident has been resolved, and all services are currently operating normally. Further investigation will be conducted to ensure robust prevention measures are in place.
WebSocket Service Disruption Affecting Chat Claiming Functionality in ProductionOperational

Incident Status

Operational


Components

Chat Mobile App


Locations

Global



November 22, 2024 12:24PM UTC
[Resolved] Overview: An issue occurred with WebSockets, resulting in the inability to claim chats. This was triggered during a deployment of the WebSocket stack to production by the development team. Root Cause: During the deployment process, a critical misconfiguration occurred. Instead of removing the non-existent CloudFormation subscription filters, the deployment inadvertently deleted all resources within the stack. Resolution: The WebSocket service has been fully restored and is now operational. However, the development team has identified the need to remove mirroring to an outdated Klarna domain to mitigate potential performance impacts. The incident was reported and fully resolved on Thursday, 20/11/2024.