ASSET: Adaptive Speech Segmentation via Reinforcement Learning
Key points
- ASSET partitions encoder outputs into variable-duration segments, pooling each into one continuous audio token.
- Segmenter is trained with GRPO using task performance feedback and a token-rate penalty.
- On English LibriSpeech ASR, reportedly matches a transcript-assisted character-boundary oracle with about 25% fewer audio tokens.
- On Mandarin AISHELL, reportedly beats a phone-boundary oracle at a similar token rate.
Why it matters for marketers: Speech LLM builders may get shorter audio inputs, cutting decoder cost, if the reported token savings hold on their own data.