Models/Speech Recognition/SraVaani 1.0

ASRMultilingual · 65 languages

SraVaani 1.0

A 430 million parameter FastConformer with a hybrid TDT–CTC decoder, pretrained from scratch on the Vaani corpus and fine-tuned to recognise speech across 65 Indian languages and dialects. That includes Tulu, Wancho, Kokborok and Bearybashe, languages no commercial ASR system covers at all.

It is the first model in the family trained end to end on Indian speech rather than adapted from a model built elsewhere, and it is released under the MIT licence with weights, ONNX export and fine-tuning scripts all public.

Methodology

How SraVaani was built

Three stages, each addressing a different problem: learning Indian acoustics from nothing, grounding those representations against meaning, and then teaching the model to write down what it hears.

131,255 hr

Pretraining from scratch

Trained from random initialisation on 31,255 hours of Vaani speech spanning 105 languages. Because nothing was inherited from a model built on other languages, the acoustic representation is shaped by Indian speech from the very beginning rather than adapted to it afterwards.

211 M pairs

Audio–image alignment

The model was then aligned against 11 million audio–image pairs. Vaani’s field collection captured speakers describing images in their own words, so these pairs carry a multimodal grounding signal that audio alone cannot provide.

3~31,270 hr

Supervised fine-tuning

Finally fine-tuned on roughly 31,270 hours of transcribed speech across 65 Indian languages and dialects, combining the Vaani transcription subset with several open-source speech corpora.

Why pretraining from scratch matters. Most Indic ASR work starts from a model trained on English or on a broad multilingual web crawl, then adapts it. That approach inherits assumptions about phonetics and prosody that do not hold across Indian languages. Starting from the Vaani corpus means the representation is built for the phonetic range it actually has to handle.

Coverage

65 languages and dialects

Coverage spans every major language family in India, plus a long tail of regional varieties that rarely appear in speech systems at all. The pretraining corpus covered 105 languages; this release is fine-tuned for 65 of them.

Scheduled languages (20)

AssameseBengaliBodoDogriGujaratiHindiKannadaKonkaniMaithiliMalayalamManipuriMarathiNepaliOdiaPunjabiSanskritSantaliSindhiTamilTelugu

Regional and low-resource languages and dialects

AngamiAoAwadhiBajjikaBearybasheBhiliBhojpuriBundeliChakhesangChakmaChhattisgarhiEnglishGarhwaliGaroGondiHalbiHaryanviIdu MishmiKarbiKhariboliKhorthaKokborokKurukhMagadhiMalvaniMarwariMizoNagameseNyishiRajasthaniRengmaRongmeiSadriSambalpuriSumiSurgujiaSurjapuriTaginTuluWancho
Urdu and Kashmiri are not supported in SraVaani 1.0, despite appearing in the pretraining corpus.

Evaluation

Benchmarks

Word Error Rate, lower is better, measured across CommonVoice, Fleurs, GramVaani, IndicTTS, Kathbath, MUCS, RESPIN and the Vaani test set. We publish the full picture, including where the model still struggles.

Under 25 WER25 to 35 WERAbove 35 WER
LanguageAvg WERCommonVoiceFleursGramVaaniIndicTTSKathbathMUCSRESPINVaani
Hindi12.412.710.920.939.19.114.610.810.8
Gujarati19.0–18.2–18.414.926.6–16.8
Assamese19.422.618.8–––––16.9
Marathi19.716.618.1–14.217.822.016.133.4
Bengali19.814.115.2–26.212.3–26.924.1
Punjabi20.229.315.9––13.6––22.1
Telugu25.1–23.1–20.921.326.425.433.5
Odia25.729.420.3–18.721.421.4–43.2
Tamil26.228.928.1–21.123.324.1–32.0
Kannada27.4–21.4–17.416.6–27.454.1
Maithili27.5––––––20.234.8
Malayalam27.733.519.9–19.431.5––34.1
Nepali31.345.826.6–––––21.5
Sanskrit36.4––––36.4–––
Manipuri41.6–––––––41.6
Konkani54.7–––––––54.7
Santali57.3–––––––57.3
Sorted by average WER. A dash means the language is not present in that benchmark.

Evaluation set and leaderboard: Vaani-Benchmark v1.0 ↗

Interactive

Try it in the browser

Inference runs on Hugging Face Spaces, not on this server. The demo below is the official ARTPARK-IISc Space embedded in the page. Record or upload a clip in any supported language and the transcript appears inline.

SraVaani demos
Open on Hugging Face ↗
Demo hosted by ARTPARK-IISc on Hugging Face Spaces. If the Space is asleep, give it a few seconds to spin up or open it directly on Hugging Face. The streaming SraVaani live Space requires sign-in and so is linked rather than embedded.

Quickstart

Start using SraVaani

The model loads through transformers with trust_remote_code. Access is gated on Hugging Face, so accept the conditions on the model card, then use your access token.

# pip install transformers torch soundfile
import torch
from transformers import AutoModel

REPO = "ARTPARK-IISc/SraVaani-1.0"
DEV  = "cuda" if torch.cuda.is_available() else "cpu"

model = AutoModel.from_pretrained(
    REPO, trust_remote_code=True, token="<hf_token>"
).to(DEV).eval()

hyps = model.transcribe("sample.wav", return_hypotheses=True)
for h in hyps:
    print(h.text)

Fine-tune on your own data

The SraVaani repo includes scripts to download the base checkpoint and run fine-tuning with NVIDIA NeMo on custom Indian-language audio, plus a notebook demonstrating an example run. Checkpoint access and request links are in the README.

Deploy with ONNX

An ONNX export with a sample inference script, for production and edge deployment where a PyTorch runtime is not practical.

Training data

The corpora behind the model are public: the full Vaani dataset and the transcription subset used for supervised fine-tuning.

Honest caveats

Limitations

Uneven data across languages

The amount of fine-tuning data varies significantly by language. For several low-resource languages the available training data is limited, which produces materially lower recognition accuracy than the high-resource languages. The benchmark tables report this openly rather than averaging it away.

Two languages unsupported

The model does not support Urdu or Kashmiri. Both appear in the 105-language pretraining corpus but are excluded from the 65 languages this release was fine-tuned for.

Gated access

The repository is publicly visible, but you must accept the conditions and share contact information on Hugging Face before downloading weights. Fine-tuning checkpoints are requested separately through the GitHub README.

Citation

Citing this work

If you use SraVaani or the Vaani dataset in your research, please cite:

@misc{pulikodan2026vaanicapturinglanguagelandscape, title={VAANI: Capturing the language landscape for an inclusive digital India}, author={Sujith Pulikodan and Abhayjeet Singh and Agneedh Basu and Nihar Desai and Pavan Kumar J and Pranav D Bhat and Raghu Dharmaraju and Ritika Gupta and Sathvik Udupa and Saurabh Kumar and Sumit Sharma and Vaibhav Vishwakarma and Visruth Sanka and Dinesh Tewari and Harsh Dhand and Amrita Kamat and Sukhwinder Singh and Shikhar Vashishth and Partha Talukdar and Raj Acharya and Prasanta Kumar Ghosh}, year={2026}, eprint={2603.28714}, archivePrefix={arXiv}, primaryClass={eess.AS}, url={https://arxiv.org/abs/2603.28714}, }

Explore the rest of the family

SraVaani is one of 20 open source speech models built on Vaani, alongside streaming recognition, speech synthesis and spoken language identification.