Next-word prediction has been put forward as a “computational principle” that might be shared across humans and machines (e.g. Goldstein et al 2022). But what are the consequences of this principle? The talk argues that this principle, in combination with freely-available fMRI data, points toward a conception of word-by-word language understanding as hierarchical structure-building. It exemplifies how modern artificial intelligence techniques allow us to specify mechanistic models of human language processing that are linguistically interpretable. Using these kinds of models, it becomes possible to address the question of whether the human parser considers one structure at a time — or more than one. 

Goldstein, A., Zada, Z., Buchnik, E. et al. Shared computational principles for language processing in humans and deep language models. Nat Neurosci 25, 369–380 (2022).

Hale, J. T., Campanelli, L., Li, J., Bhattasali, S., Pallier, C., & Brennan, J. R. (2022). Neurocomputational Models of Language Processing. In Annual Review of Linguistics (Vol. 8, Issue Volume 8, 2022, pp. 427–446). Annual Reviews.

Stanojević, M., Brennan, J.R., Dunagan, D., Steedman, M. and Hale, J.T. (2023) Modeling Structure-Building in the Brain With CCG Parsing and Large Language Models. Cognitive Science, 47: e13312.