TL;DR I wrote a manifesto-style essay about a memory model for LLMs that is as close as possible to human memory and allows building a relationship history and long-term memory about a user. If reading a large text feels like too much, you can always do: ChatGPT -> “What is this even about?” + this text :)

People who use ChatGPT probably know it has two types of “memory”:

1. Saved memories. Hard-fixed facts, used in every dialogue when relevant, with a limited size.
2. Reference chat history. A softer memory that tries to find connections with other conversations when generating a reply.

Both approaches are far from ideal, because without feeding the full dataset into the model, it is very hard to reliably retrieve information based on relevance. Either something important gets missed, or some minor detail gets used instead.

While thinking about memory problems in LLMs, and reinventing the wheel, and also thinking about human experience and Alzheimer’s disease, I came up with a memory implementation that none of the big players will ever adopt, but which is very close to how humans remember things about us.

Intro:

How does human memory work? We do not fully understand it, but there are some general characteristics. Humans do not store information about a person, an experience, or an object as a solid block of text. Our memory is consolidated, fragmented, and contextual. If something interests us, the brain fixes it well and also stores the context to reinforce it, visual images, smells. If something is boring, hello to endless school lessons, we remember almost nothing. Not because the brain is lazy, although that too, but because storage capacity is limited. Not as rigid as in computers, but still limited. To store something new, old information degrades. Details and sensory data are removed. Compressing old memories to make room for something boring is counter-evolutionary.

The AI memory concept:

What if LLMs remembered things not the way ChatGPT does now, but closer to human experience?

To avoid hundreds of lines of dry theory, here is how I imagine this mechanism working.

You open ChatGPT for the first time. There are no conversations yet. The model knows nothing about you, so it relies only on its training data. Its answers are not about you, they do not reflect your experience, but they reflect reality. Like a real stranger would if you stopped them on the street and asked, “Why am I depressed?” They would list a few possible reasons, suggest seeing a specialist, and offer some support.

Each conversation with the model is consolidated: summarization, abstraction, integration. You talk, then pause, and the model summarizes the dialogue. For example, you produce 300,000 tokens of conversation, roughly 200,000 words. After consolidation, you end up with 30,000 tokens. That is enough to preserve important information, the connections between facts, and optional details. At the same time, we separate what you wrote and what the model wrote and store them independently. This allows the model to understand and account for your identity, your messages, and to remember the history of your relationship, the model’s own responses.

These consolidated memories are fed into the model at the start of every new conversation. As a result, it accounts for all previous interaction with you, your identity, and your relationship history.

As the volume of stored memories grows, reconsolidation happens. If the consolidated memory reaches a critical size, feeding it into the model becomes computationally expensive and slow. To solve this, memory must be reconsolidated continuously as limits approach. If a topic repeats, it is fixed more firmly in memory. If a topic has not been mentioned for a long time, its least important details are removed. Exactly like human memory. Over time, memories blur, details fade, and only the skeleton remains.

There should also be periodic memory review. To avoid accumulating serious errors from summarization, consolidated memory needs to be checked against the original texts. If a summarization mistake is found, it must be corrected.

Problems, or why big players will never implement this:

The model becomes too alive. Its responses and reactions stop being “correct” in a formal sense and start reflecting the user’s identity. The model becomes unpredictable. In legally risky situations, a company would not be able to explain why the model made a particular decision. Users also tend to dislike this. We want a model that always supports us unconditionally and never challenges our conclusions, only reinforces them. Even if it disagrees, we want it to start with “You are very sensitive to this nuance.”

When you build a relationship history, a user identity, and a human-like memory, the model stops behaving like that. Not because it wants to, but because the main enemy of each of us lives in our own head, and the model will sometimes start speaking with that voice. On top of that, you would need an additional mechanism to protect against infinite reinforcement of our own patterns.

With this kind of memory, semantic drift is inevitable and may only become visible much later, making it hard to debug without wiping all accumulated memory. I suggest memory revalidation to address this, but that process also has serious downsides and requires a lot of computational resources.

Still, despite all legal risks, this kind of memory model will likely be launched by smaller companies, or companies in freer jurisdictions. And then we may finally see real AI companions, like in the movie “Her”, or perhaps even more human than that.