Mathpix - Convert Image API Disruption – Incident details

All systems operational

Convert Image API Disruption

Resolved
Operational
Started 7 months agoLasted about 2 hours

Affected

OCR API (us-east-1)

Partial outage from 12:45 PM to 2:45 PM

Updates
  • Resolved
    Resolved

    Our engineering team quickly identified the issue and implemented a fix to restore normal service operations. We conducted a thorough investigation to pinpoint the root cause and have developed long-term solutions to strengthen our system:

    • Autoscaling Improvements: We've revised our autoscaling policies and algorithms to better handle unexpected usage behaviors and high-demand scenarios.

    • Enhanced Monitoring: Implementation of advanced monitoring tools to detect anomalies in real-time.

    • Resource Allocation: Additional resources have been allocated to ensure robust performance during traffic surges.

  • Identified
    Identified

    A service disruption was caused by an error in our autoscaling policy. Unexpected usage patterns from some of our top customers revealed flaws in our autoscaling algorithm. This led to service degradation to our Image Convert API endpoints for customers routed to our US data centers, while customers closer to Europe or Asia were unaffected.