Research

Speech language models as models of listening

In psycholinguistics research, surprisal is almost always computed from text, even in studies where people are listening to spoken input. We develop a framework to estimate it directly from the audio with speech language models. Doing so raises questions about units: one word has many acoustic realizations, each assigned a different probability by a given speech LM. Using importance sampling, we marginalize over realizations generated by a text-to-speech model and study this estimated marginalized surprisal as a predictor of pupil dilation in listening studies.

Estimating Surprisal from the Speech Signal (to appear at EMNLP 2026).

Getting the predictors right

Reading-time studies usually take contextual surprisal from a language model and word frequency from a separate corpus. We show how to estimate both from the same model. We test whether downstream models of eye-tracking data in 11 languages change when unigram and contextual surprisal come from the same model.

Explore the unigram surprisal results

Every model comparison from Revisiting the Estimation of Unigram Surprisal in Computational Psycholinguistics (to appear at EMNLP 2026), across eleven MECO languages and three English corpora. Switch estimators, spillover, outlier removal, log-transformed reading times and multiple-comparison correction.

Speech and text technology for language varieties

Dialect robustness in Catalan speech recognition, parallel data and a lemmatizer for the written varieties of Romansh, orthographic variation in Occitan, and low-resource speech translation. See publications.