BanglaDialecto: An End-to-End AI-Powered Regional Speech Standardization

✅ Accepted at IEEE International Conference on Big Data 2024 · (* Equal Contribution)

📄 Paper: arXiv:2411.10879
💻 Code & Dataset: github.com/EncryptedBinary/BanglaDialecto

Abstract

This project addresses recognizing Bangladeshi dialects and converting diverse Bengali regional accents into standardized formal Bengali speech. We develop a large Noakhali Dialect Dataset (NDD) — 10 hours of regional speech with parallel standard Bangla transcriptions — and design a full end-to-end pipeline with three stages: (1) dialect speech recognition (ASR), (2) dialect-to-standard Bangla machine translation (MT), and (3) standard speech synthesis (TTS). Fine-tuned Whisper Large V2 achieves CER 0.8%, WER 1.5% on dialect ASR; BanglaT5 achieves BLEU 41.6% for dialect→standard translation. AlignTTS completes the pipeline with high-quality Bangla speech synthesis.

Introduction

Bangladesh has diverse regional dialects (Noakhali, Sylheti, Chittagonian, Mymensingh, etc.) that differ significantly from standard formal Bangla in phonology, vocabulary, and grammar. Speakers face barriers in voice-assisted services, transcription tools, and communication systems built only for standard Bangla. Prior work on Bangla dialect standardization is limited by small datasets and pipeline fragmentation.

We build the first end-to-end AI pipeline for Noakhali dialect standardization, chaining state-of-the-art ASR → MT → TTS models, all fine-tuned on purpose-built dialectal data.

Task Formulation: For n dialect speech signals {s^i}, each with dialect text t_d^i and standard text t_s^i:

• Model ℱ₁: transcribe s^i → t_d^i (Dialect ASR)
• Model ℱ₂: translate t_d^i → t_s^i (Dialect MT)
• AlignTTS: generate standard speech from t_s^i (TTS)

🎯 Key Contributions

• Noakhali Dialect Dataset (NDD) — 10 hours of annotated dialect speech (7,200 samples after segmentation) with parallel standard Bangla translations; collected from YouTube, Facebook, and direct interviews with 24 native speakers.
• End-to-End Dialect Standardization — ASR + MT + TTS chained into a seamless pipeline.
• Whisper Fine-tuning — CER 0.8%, WER 1.5% on Noakhali dialect — significantly outperforming all prior Bangla ASR baselines.
• BanglaT5 Fine-tuning — BLEU 41.6% for dialect→standard translation, surpassing mBART50, IndicBART, mT5.

Methodology

Figure: NDD dataset statistics. 24 participants (75% male, 25% female), age 18–48, with audio collected at 16 kHz mono, 16 kHz sample rate.

III-A. NDD Dataset Construction

Collection: 10 hours of Noakhali regional speech from YouTube, Facebook, and interviews. Participants read standard paragraphs in their native Noakhali accent across diverse sub-regions.

Dataset Properties:

• Unique Characters: 72
• Total Unique Words: 11,246
• Audio: 10 hours, 16 kHz, mono, 16-bit

Annotation: Qualified human evaluators from Noakhali annotated dialect transcriptions and standard Bangla translations in a 3-column format: (s^i, t_d^i, t_s^i).

III-B. Preprocessing

1. Denoising: Background noise removed using noisereduce library
2. Segmentation: Each signal split into fixed 5-second segments
   N^i = ⌊T^i/5⌋ segments per recording
3. Text Alignment: Dialect and standard text chunked to match speech segments

Post-segmentation split:

III-C. Model Pipeline

Stage 1 — Dialect ASR (Whisper fine-tuning)

• Input: dialect audio segments s^i_k
• Output: dialect text t_d^i_k
• Training: 10 epochs, batch size 16
• Models: Whisper-base (74M) to Whisper-Large V2 (1550M)

Stage 2 — Dialect MT (BanglaT5 fine-tuning)

• Input: dialect text t_d^i_k
• Output: standard Bangla t_s^i_k
• Training: 25 epochs, batch size 6
• Models: XLM-ProphetNet, mBART50, IndicBART, mT5, BanglaT5

Stage 3 — TTS (AlignTTS)

• Input: standard Bangla text
• Output: synthesized speech
• AlignTTS replaces attention with alignment loss + dynamic programming for stable Bangla synthesis

📊 Table I — NDD Dataset Details

📊 Table II — Comparison with Prior Dialect Standardization Work

ASR (Dialect Speech → Dialect Text):

MT (Dialect Text → Standard Text):

📊 Table III — Pre-trained vs Fine-tuned Model Performance

Dialect ASR (Whisper Models):

Dialect MT (Translation Models):

📚 Citation

@inproceedings{samin2024bangladialecto,
  title={BanglaDialecto: An End-to-End AI-Powered Regional Speech Standardization},
  author={Samin, Md Nazmus Sadat and Ahad, Jawad Ibn and Medha, Tanjila Ahmed and Rahman, Fuad and Amin, Mohammad Ruhul and Mohammed, Nabeel and Rahman, Shafin},
  booktitle={IEEE International Conference on Big Data (BigData)},
  year={2024}
}