CAFE confines feature distillation to adapters to improve compact multilingual speech recognition
Key points
- Study finds feature distillation does not add to logit distillation for two common feature losses in compact multilingual ASR.
- CKA loss lowers WER on non-English languages but hurts data-rich English, per the paper's gradient analysis.
- CAFE routes the CKA gradient into adapters English bypasses, cutting average WER from 8.55% to 8.02% on Open ASR Leaderboard.
- At 254M parameters, CAFE trails its 0.6B teacher by 2 WER points across 48 test sets in 24 languages.
Why it matters for marketers: Teams distilling compact multilingual speech models may get non-English gains without English regressions by isolating feature losses in language-specific adapters.