Keka GenAI Chatbot: Deploying AI Support at Scale
Shipped a GenAI support chatbot from zero to production by building stakeholder trust through evals and daily metrics.
Bhargava T
AI Product Manager at SquadStack.ai
From their time as
Associate Product Manager
Keka HR • 2023 - 2025
Overview
Bhargava took on the GenAI chatbot at Keka HR when conversational AI was still a new concept internally. The challenge was not the model; it was getting the customer support team to trust it enough to give him the bandwidth to run proper evals.
The Story
Bhargava took on the GenAI chatbot at Keka HR when conversational AI was still a new concept internally. The challenge was not the model; it was getting the customer support team to trust it enough to give him the bandwidth to run proper evals.
He started by demonstrating what the bot could actually do. It handled informational queries well. It struggled with troubleshooting and compliance-sensitive questions, and he was transparent about both. He showed the support leader exactly where it worked and where it did not, and that honesty built the initial conviction.
Once the support leader committed, Bhargava was given two subject matter experts from the support team. He used them to cover edge cases across Keka's eleven products, starting with one product and scaling from there. He ran A/B tests comparing human responses to bot responses, iterated on prompts and FAQs, and maintained a weekly and bi-weekly cadence with the support leadership throughout.
The bot went live for the first 200 customers, then scaled. Within the first quarter, it reached a 53% support ticket deflection rate and was handling two thousand queries per day.