Bangla Speech-to-Text Benchmark 2026 Which speech-to-text API understands Bangla best?

We randomly sampled 77 clips from each of 13 domains of BanSpeech (SUST CSE Speech Group, CC BY-NC 4.0), excluding regional dialects. Clips are 16 kHz WAV, 0.7 to 26.8 s long (median 2.1 s), 50.0 minutes in total, with 7,996 reference words and 43,411 characters.

• CER (primary) and WER are computed with jiwer at corpus level: total edits across all clips divided by total reference length, not an average of per-clip rates.
• 95% confidence intervals come from 10,000 bootstrap resamples of clips. Comparisons between providers use a paired bootstrap over the clips both transcribed.
• Failed requestsare excluded from that provider’s score rather than counted as 100% error. Every provider covered at least 99.4% of clips.

The same normalizer runs on references and transcripts before scoring, so formatting choices don’t count as errors:

1. Unicode NFC normalization.
2. Remove zero-width characters (ZWNJ, ZWJ, BOM).
3. Map Bangla digits ০–৯ to 0–9.
4. Replace punctuation (including the danda ।) with a space.
5. Lowercase Latin text, then collapse whitespace.

Source: src/normalize.py

• Batch: one request per clip, one at a time. Latency is wall-clock request time; real-time factor (RTF) is latency divided by audio length.
• Streaming: 16-bit PCM sent in 100 ms chunks, each only after that much real time had passed, exactly like a live microphone. Accuracy is scored on the final transcript.
• Settings:provider defaults apart from the language. No custom vocabulary, no fine-tuning. Only Gemini batch received a prompt (“Transcribe this Bengali (Bangla) audio verbatim…”).
• Environment: a MacBook in Bangladesh on an ordinary internet connection. Provider servers were in the US, Europe or India, so every latency includes that round trip. Chirp 2 ran in us-central1.

List prices were checked on 24 September 2026 from each provider’s public pricing page. “Effective” price applies each provider’s billing rules (per-second rounding, minimum billable durations, token conversions) to these exact clips. Groq’s 10-second minimum, for example, makes short clips cost 3.4× its list price. Sarvam’s INR price is converted at ₹95.96 per US$. Every rate and its quoted source text is in pricing/prices.yaml.

• A snapshot of these APIs in September 2026. Providers update models often.
• Mostly read and broadcast speech. No phone audio, heavy noise, dialects or Bangla-English code-switching.
• Language tags differ by provider (bn, bn-IN, bn-BD, ben), which may affect results.
• Gemini Live returned no interim transcripts, and its adapter set no language hint.
• ElevenLabs batch requested scribe_v1, which was scheduled for removal; every request succeeded, so it’s unclear which version served them.
• Only hosted APIs were tested, not self-hosted open-weight models.

Riverborn Limited funded and ran this benchmark. We build AI and voice products, and have no commercial relationship with any provider tested. We publish all code and outputs so the results can be checked independently.

In our September 2026 test, Sarvam AI (saarika:v2.5) was the most accurate for recorded audio at 6.9% character error rate, and Soniox (stt-rt-v5) was the most accurate for live streaming at 7.4%. Soniox batch was a close second at 8.1% and the cheapest option tested.

Soniox (stt-rt-v5) was the most accurate streaming API at 7.4% CER, statistically ahead of Sarvam AI (9.0%). Sarvam returned the final transcript fastest after the speaker stopped (0.9 s median vs 2.5 s for Soniox), so it may suit voice agents that need a quick reply.

Soniox was the cheapest in both modes: $0.10 per audio hour for batch and $0.17 for streaming, with the second-best batch accuracy (8.1% CER). Google Cloud Speech-to-Text (Chirp 2) was the most expensive at $1.12 per hour. Prices were checked on 2026-09-24.

Chirp 2 (language bn-BD) scored 21.9% CER in batch and 23.6% in streaming, ranking 6 of 8 in batch. Google Gemini 2.5 Flash did better in batch at 13.3% but poorly through the Live API (69.6%).

Not on this test. Whisper large-v3 (served by Groq) scored 30.7% CER and 71.7% WER, the lowest accuracy of the 8 batch APIs and about 4.5× the error rate of the leader. It was the fastest to respond (0.66 s per clip), and Groq has no streaming API.

Use whatever the provider supports. In this benchmark Sarvam used bn-IN, Google Chirp 2 used bn-BD, ElevenLabs used ben, and Deepgram, OpenAI, Groq and Soniox used bn. The audio is standard Bangla from BanSpeech (SUST, Bangladesh); we did not test whether switching codes changes accuracy.

Character error rate (CER) is the number of character substitutions, deletions and insertions needed to turn a transcript into the reference, divided by the reference length. A CER of 7% means roughly 7 wrong characters per 100. Word error rate (WER) is the same at word level. We rank on CER because Bangla word spacing varies between annotators and engines, which inflates WER.

On this test set, OpenAI gpt-4o-transcribe scored 28.7% CER and Whisper large-v3 (via Groq) 30.7%, about 4 to 5 times the error rate of the leaders.

Riverborn Limited funded and ran the benchmark. We have no commercial relationship with any provider tested. All code, prompts, settings, per-clip transcripts and scores are public so anyone can check or rerun it.

No. The audio is BanSpeech: mostly read and broadcast-style standard Bangla in 13 domains. Regional-dialect clips were excluded, and there are no phone calls, heavy noise or Bangla-English code-switching. Test on your own audio before choosing.

Yes. The repository has the test set definition, provider adapters, scoring and confidence-interval code under the MIT license. Add your own API keys and run the scripts; the README walks through each step.