A study by Chibuzor Okocha and Christan Earl Grant tests speech systems on English-Yoruba code-switching: moving between languages within speech. The authors' arXiv paper, submitted on 10 September, evaluates 11 systems on 2,000 utterances.
The researchers compare six automatic speech recognition models and five audio language models. They find that overall word error rate can conceal differences at the points where speakers change language.
English tokens are recognised more reliably than Yoruba tokens in the tested systems. Some generative models translate or add material instead of returning an exact transcript.
These findings concern the study's evaluation set and tested systems, rather than every African language or transcription product. For editors, they support checking multilingual transcripts against the audio instead of relying on a single overall accuracy score.