Building Voice Agent Infrastructure at a Stealth AI Startup
Owns the full voice agent stack, from ElevenLabs integration to backend and frontend, while solving latency and scaling challenges
Manish Khetan
Software Developer at Stealth AI Startup
From their time as
Software Developer
Stealth AI Startup • 2025
Overview
Manish joined a stealth AI startup three months ago as a software developer and took full ownership of the voice agent stack, covering voice synthesis integration, backend infrastructure, and frontend.
The Story
Manish joined a stealth AI startup three months ago as a software developer and took full ownership of the voice agent stack, covering voice synthesis integration, backend infrastructure, and frontend.
The core problem the team is solving is understanding what people are actually doing, using voice agents as the primary interface. Manish owns the entire technical implementation, from how the AI speaks and is prompted to how calls are screened and processed.
The team is working toward significantly higher concurrency than current third-party integrations allow, which will eventually require building proprietary voice synthesis infrastructure. Latency and scaling are the two active engineering challenges he is working through.