Oshara · Engineering
Notes from the voice-AI trenches
From-the-trenches notes on building voice AI at Oshara — the bugs, the fixes, and what we learned shipping them.
Fine-Tuning IndicConformer for Nepali Speech Recognition
We benchmarked AI4Bharat's IndicConformer against six other Nepali STT systems, then fine-tuned it further — hitting a tokenizer bug along the way that taught us something useful about training data hygiene.
Two LLMs Are Better Than One: How We Build Low-Latency Voice Agents
The awkward pause in voice AI is a timing problem, not an intelligence problem. This is the two-parallel-LLM architecture we use to fix it — a fast model that buys presence, a smart model that carries the answer — plus a tour of how a real-time voice agent is actually assembled.
A Nepali Voice for XTTS v2
Nepali has no open, zero-shot text-to-speech system. We built one by fine-tuning XTTS v2 — reusing its existing Hindi tokenizer instead of adding a single new vocabulary token — and it's now the best open, self-hostable Nepali voice we could measure.
Building an Agent, Step by Step
A complete walkthrough of creating a voice agent on Oshara.ai — from the My Agents page all the way to a deployed, embeddable widget.
Teaching a Duplex Voice Model to Call Tools
PersonaPLEX speaks and listens at the same time, but it can't look anything up. This is how we grafted a tool-calling action stream onto a full-duplex audio model — the architecture changes, the four training attempts, and why it kept searching for the wrong thing.
Teaching PersonaPLEX to Recite Shakespeare
How we injected new knowledge into a full-duplex speech model with LoRA — turning a Shakespeare play into spoken question-answer data, and fine-tuning PersonaPLEX in document mode to recite grounded lines and attributions out loud.
Teaching XTTS v2 to Speak Nepali
How we fine-tuned XTTS v2, a zero-shot voice-cloning TTS model, into a fluent Nepali narrator: the architecture, why epoch 10 beat epoch 30, how it stacks up against a different TTS model, and the text-pipeline fixes that stopped it from babbling.
Teaching a Machine to Speak Nepali
A from-the-trenches account of fine-tuning Chatterbox v3, a voice-cloning TTS model, for Nepali — the tokenizer bugs, the phantom sentences, the overfitting hunt, and the audio that came out the other side.