AssemblyAI announced that its speech‑to‑text inference service now operates on Amazon ECS and EC2 Spot GPU instances, delivering a 70% reduction in compute costs compared with On‑Demand pricing while preserving service level agreements.
The solution is deployed in the US West (Oregon) and Europe (London) AWS regions, with each region hosting independent ECS clusters, SQS queues and GPU fleets to satisfy data‑residency requirements for customers in healthcare, finance and legal sectors.
Cost‑Optimised Architecture
AssemblyAI’s architecture combines On‑Demand and Spot EC2 instances across NVIDIA GPU families g4dn, g5 and g6. Amazon ECS capacity providers dynamically allocate compute resources, using On‑Demand instances as a baseline and scaling with Spot instances during demand spikes.
Spot instance interruptions are managed by a Lambda‑based Spot Interruption Handler that returns in‑flight jobs to an Amazon SQS queue for retry. Jobs that exceed the two‑minute interruption window are re‑queued and processed by another healthy ECS task, minimising service disruption.
Rapid Recovery and Reliability
Custom AMIs containing model artefacts and container images are prepared in under two minutes, enabling quick recovery after interruptions. Diversifying across GPU instance families reduces interruption rates and improves overall reliability during peak demand.
Amazon CloudWatch monitors fleet health and Spot interruption events, providing system observability and ensuring that the platform sustains unit economics even as compute demand grows nearly four times year‑over‑year.
The platform’s deployment in two separate AWS regions ensures compliance with geographic data‑retention laws, a critical requirement for regulated industries.