ľ¹Ï¸£ÀûÓ°ÊÓ

Short description

To be able to talk and to understand each other, we have to continuously store and retrieve linguistic information. In linguistics, the dominant approach to studying the processes of storing and recall of linguistic information from short-term memory assumes that we can access all items in parallel and that the most highly activated items are the most likely to be retrieved. Activation, in turn, can be boosted by the requirements of the current cognitive context.

This model is related to theories of memory developed independently of linguistics. In linguistics, it has been supported by rich research on production and comprehension. The model, however, has been applied very narrowly. It focuses only on the recall of some syntactic items, for instance, the recall of arguments during the processing of a verb. Other functions of memory fall outside the approach.

The project’s core idea is that the memory model can be applied to many other cases in which memory has a decisive role. We will do this by linking the model to theories of other language phenomena developed in linguistics, cognitive sciences and artificial intelligence. First, we will link it to computational models of lexical knowledge, which will enable us to fully and formally represent what the current cognitive context is and to build an indiscriminate and general approach to memory access. Second, we will link it to computational models of grammatical knowledge to understand how we store and recall grammatical rules. Finally, we will link it to discourse theories to have an analysis of storage and recall of textual information.

The project will lead to a new view on the memory model, one that is general and cross-domain. It will provide a more principled account of how memory affects language, will give us a new insight into why the theories of lexical knowledge, grammatical knowledge and discourse theories work, and it will make it possible to tie together accounts that are often treated as independent.

Postdoctoral researcher in the MEMLANG project (open position, deadline November 9)

The Memory Access in Language (MEMLANG) project investigates the role of memory in communication. The project consists of three parts:

• experimental part: Very generally speaking, we investigate the properties of short-term and long-term memory in language comprehension. More concretely, we are looking at interferences and decay mechanisms that affect the correct recollection of facts and linguistic elements. We use several techniques—from reaction-time studies through eye tracking to EEG—and several types of data—from the recollection of propositional factual information to the correct interpretation of discourses.
• theoretical part: We investigate theories of associative memory and to what extent they are applicable to language. Our main foci are: (i) the ACT-R theory of spreading activation (Anderson and Lebiere, 1998), which was brought into psycholinguistics under the name of cue-based retrieval (Lewis and Vasishth, 2005); (ii) Hopfield networks, which have been recently revived as Dense Associative Memory (Hopfield and Krotov, 2016, Krotov et al., 2025).
• computational part: Our aim is to understand how associative memory is represented in the current state-of-the-art architectures, as well as how associative memory models listed in the theoretical part can be linked to and shed light on currently used architectures.

The postdoctoral position is part of the computational part of the project. The postdoctoral researcher should focus on NLP and computational cognitive modeling, but they should also be ready to make links to the theoretical part and to understand the experimental aspects of the project (they are not expected to run any experiments on their own, but should be able to follow experimental work and they might collaborate with experimental researchers during the project).

There are two main lines that the project should focus on. However, what is described below should be read as potential suggestions of directions to be investigated in the project. The specifics will be decided jointly with the postdoctoral researcher.

Interferences in LLMs

How do interferences manifest themselves in LLMs? Consider the following example: The manager ate an apple but the assistant ate a pear. Later the same day, the manager again ate a pear. Based on our experiments, we know that humans are susceptible to interferences when resolving the presupposition following again, e.g., the information about what the assistant did, can lead to a (short-lived) illusion of coherence in the discourse. This is a case of interference—semantically similar information affects the resolution of the presupposition.

Related phenomena are currently studied in NLP for in-context learning under various guises (context hijacking, contextual entrainment and distractions). Cases of interferences have also been studied by NLP language researchers who, however, mainly focused on syntax and on so-called agreement attraction phenomena. We want to see to what extent interferences across syntax and semantics have a common denominator in LLMs and to what extent they can be related to the aforementioned phenomena like context hijacking. The researcher should be ready to test and investigate LLMs and to use various techniques from mechanistic interpretability to probe into the models.

Associative memory

Interferences are particular manifestations of and windows into associative memory (of humans and LLMs). So how can we understand the behavior of LLMs through the prism of associative memory? Is it human-like? Is it not? We are particularly interested in Dense Associative Memory as a framework for associative memory (i) that is formalized and whose formal properties are well-studied, (ii) that can be linked to other architectures, like diffusion models and transformers.

In this part of the project, the researcher should focus on studying how the Dense Associative Memory model aligns with theories that have been proposed and tested in the past in psycholinguistics and cognitive science (cue-based retrieval, ACT-R spreading activation). Second, the researcher should study to what extent quantitative predictions of Dense Associative Memory can explain human behavior in reading (as glimpsed through EEG and eye-tracking reading-time data, etc.). The researcher should be ready to train and explore toy models of Dense Associative Memory, as well as explore the links between Dense Associative Memory and other models (diffusion models, transformers) to explore the properties of associative memory in current modeling research.

References

Anderson, John R. and Lebiere, Christian (1998). The Atomic Components of Thought. Lawrence Erlbaum Associates, Hillsdale, NJ.

Hopfield, John J and Krotov, Dmitry (2016). Dense associative memory for pattern recognition. Advances in neural information processing systems, 29.

Krotov, Dmitry, Hoover, Benjamin, Ram, Parikshit, and Pham, Bao (2025). Modern Methods in Associative Memory. arXiv preprint arXiv:2507.06211.

Lewis, Richard and Vasishth, Shravan (2005). An activation-based model of sentence processing as skilled memory retrieval. Cognitive Science, 29:365-419.

Recent output

Psycholinguistics:

Schmitz, Tijn, Nouwen, Rick, & Dotlacil, Jakub (2025). . Journal of Memory and Language, 143

Kloostra, Li, Nouwen, Rick, & Dotlacil, Jakub (2025) Memory Retrieval in Discourse with ‘again’: Eye-tracking and acceptability studies. AMLAP presentation.

Link, Philine, van Maanen, Leendert  & Dotlacil, Jakub (2025). Similarity Comes at a Cost: Novel Evidence for Associative Memory Retrieval. AMLAP presentation.

Schmitz, Tijn, Winkowski, Jan, Hoeks, Morwenna, Nouwen, Rick, & Dotlacil, Jakub (2024). . Glossa Psycholinguistics, 3.

Winkowski, J., Dotlacil, J., & Nouwen, R. (2024). Searching for distance effects. Glossa Pyscholinguistics, 3.

Lacina, Radim, & Dotlacil, Jakub (2024). Grammaticality illusions in Czech: A speeded acceptability study of agreement attraction. In Proceedings of the annual meeting of the cognitive science society, 46.

NLP:

Takmaz, Ece, Gatt, Albert, & Dotlacil, Jakub (2025). Traces of Image Memorability in Vision Encoders: Activations, Attention Distributions and Autoencoder Losses. ICCV 2025, workshop MemVis.

Takmaz, Ece, Bylinina, Lisa, & Dotlacil, Jakub (2025). Model Merging to Maintain Language-Only Performance in Developmentally Plausible Multimodal Models. EMNLP 2025 Workshop BabyLM