Linguistic register is defined as a variety of language shaped by different situational settings. Adapting to register is crucial for successful communication and involves the processing of language features related to register variation. Few studies have focused on the impact of linguistic register on language processing. Our research investigates whether register variation affects the detection of linguistic errors. To determine if linguistic background further impacts the way we deal with register, our sample includes monolingual, bilingual, and bidialectal participants. All groups completed an acceptability judgement task in Italian that features Subject-Verb agreement mismatches presented in high and low register. The results reveal a significant impact of linguistic register on accuracy: morphosyntactic errors are better detected in low-register stimuli. Furthermore, different trends characterize the tested groups. While monolinguals show more similar accuracy rates for low- and high-register sentences, the bilingual groups tend to better spot errors in low-register stimuli. Our findings suggest that register plays an important role in the processing of morphosyntactic errors, highlighting the need to consider both its cognitive and social dimensions. Moreover, the variation observed among the tested groups underscores that language processing can be influenced by factors related to the sociolinguistic dimensions of each linguistic community.

Modern Artificial Intelligence applications show great potential for language- related tasks that rely on next-word prediction. The current generation of Large Language Models (LLMs) have been linked to claims about human-like linguistic performance and their applications are hailed both as a step towards artificial general intelligence and as a major advance in understanding the cognitive, and even neural basis of human language. To assess these claims, first we analyze the contribution of LLMs as theoretically informative representations of a target cognitive system vs. atheoretical mechanistic tools. Second, we evaluate the models’ ability to see the bigger picture, through top-down feedback from higher levels of processing, which requires grounding in previous expectations and past world experience. We hypothesize that since models lack grounded cognition, they cannot take advantage of these features and instead solely rely on fixed associations between represented words and word vectors. To assess this, we designed and ran a novel ‘leet task’ (l33t t4sk), which requires decoding sentences in which letters are systematically replaced by numbers. The results suggest that humans excel in this task whereas models struggle, confirming our hypothesis. We interpret the results by identifying the key abilities that are still missing from the current state of development of these models, which require solutions that go beyond increased system scaling.

Understanding the limits of language is a prerequisite for Large Language Models (LLMs) to act as theories of natural language. LLM performance in some language tasks presents both quantitative and qualitative differences from that of humans, however it remains to be determined whether such differences are amenable to model size. This work investigates the critical role of model scaling, determining whether increases in size make up for such differences between humans and models. We test three LLMs from different families (Bard, 137 billion parameters; ChatGPT-3.5, 175 billion; ChatGPT-4, 1.5 trillion) on a grammaticality judgment task featuring anaphora, center embedding, comparatives, and negative polarity. N = 1,200 judgments are collected and scored for accuracy, stability, and improvements in accuracy upon repeated presentation of a prompt. Results of the best performing LLM, ChatGPT-4, are compared to results of n = 80 humans on the same stimuli. We find that humans are overall less accurate than ChatGPT-4 (76% vs. 80% accuracy, respectively), but that this is due to ChatGPT-4 outperforming humans only in one task condition, namely on grammatical sentences. Additionally, ChatGPT-4 wavers more than humans in its answers (12.5% vs. 9.6% likelihood of an oscillating answer, respectively). Thus, while increased model size may lead to better performance, LLMs are still not sensitive to (un)grammaticality the same way as humans are. It seems possible but unlikely that scaling alone can fix this issue. We interpret these results by comparing language learning in vivo and in silico, identifying three critical differences concerning (i) the type of evidence, (ii) the poverty of the stimulus, and (iii) the occurrence of semantic hallucinations due to impenetrable linguistic reference.