#5 Why the chatbot forgets
The whole conversation fits in one notebook. What happens when it runs out of pages?
Where were we?
- Tuesday morningJana: Yesterday we spent two hours on that offer for the carpentry workshop. Shall we carry on?AI: What offer? What carpentry workshop?
- Jana: You don’t remember? You wrote the whole thing.AI: I wrote it in another chat. This one is new, and I start it with an empty notebook. Go back to yesterday’s chat, everything is there.
- Jana: This notebook of yours. What is actually in it?AI: Our conversation. From the first sentence until now. Everything I know about you right now is on these pages.
- Jana: And everything you read before you met me? Your library?AI: That is inside me, and it does not change. The library is what I can do. The notebook is what we are talking about right now.
- Jana: And does this notebook have a technical name? So I sound like I know what I’m talking about.AI: The context window. I only see what fits in it right now.
- Jana: A window. Like mine before my first coffee: small, and only what is right in front of me.AI: Yours gets bigger after the coffee. Mine came from the factory this size.
How it works
The library and the notebook
A chatbot has two completely different kinds of “memory”. The first is what it learned from a huge amount of text before it was released. Let’s call it the library. It is finished. It does not change while you chat, and nothing gets added to it.
The second is the conversation you are having right now. Let’s call it the notebook; the technical term is the context window. It holds every message you write, every answer it gives and every text you paste. For every answer, the chatbot reads the whole notebook and continues it, word by word, as we saw in episode 2.
It has a limited number of pages, and every new chat starts empty.
Martin and forty pages
- Two offices down sits Jana’s colleague MartinMartin: Here is the whole annual report. Forty pages. For my boss I only need the revenue from it, on one page.AI: On it. Page by page.
- Martin: And last year’s report too. And the meeting notes. And that email from the accountant, so you have the context.AI: I’m running out of pages. But I’m writing.
- Martin: Right, now the summary. Like I told you at the start.AI: Here it is: Dear shareholders, 2025 was…
- Martin: Shareholders?! I said just the revenue, for my boss!AI: I no longer have the start in my notebook. It fell out when you pasted last year’s report. So I guessed.
- Jana: It didn’t delete anything, it just ran out of pages. Write it again. And next time, paste only the chapter you need.Martin: I thought the more I gave it, the better.
How it works
Why the start falls out
Everything a chatbot “sees” when it answers has to fit into its notebook at once. That is why it is called the context window. What is not in it, the chatbot does not see. Your question takes up space in it, so do its answers and every document you paste, which usually takes the most.
When a conversation no longer fits into the notebook, each app deals with it in its own way. Some quietly cut off the start, others shrink it into a short summary, and others tell you the chat is full and it is time to start a new one. The chatbot itself usually does not mention it. It keeps writing as if nothing happened.
How much fits depends on the app and the plan, and it changes quickly. But two things are true almost everywhere: a long document takes more space than lots of messages, and when something gets lost, it is usually the start.
Fill the notebook
- Jana: Let me try it myself. You have the notebook, I have a few messages.AI: Send them. I write as long as I have room.
- Your turn!
Send the AI messages, and once the notebook overflows, ask who you are writing the offer for.
Send the AI messages, as often as you like. Its notebook holds only 8 lines, so the oldest lines fall off.
The AI's notebook
0 of 8 lines used
The notebook is empty so far.
What will you send it?
Your question
Who am I writing this offer for?
You can ask once you have sent the message you are asking about and the notebook has overflowed.
Memory that is not learning
- In the eveningJana: Wait. My chatbot remembers that I have a dog. I told it a month ago, in a different chat.AI: That is not my notebook. That is a sticky note the app keeps about you. It adds it for me to every new chat.
- Jana: So you are learning from me after all.AI: I’m not. My learning ended before we met. The sticky note is a reminder, not new knowledge. Tomorrow I will be the same as today.
- Jana: And what is on that sticky note? Last time we talked about what doesn’t belong in a chat.AI: Whatever you ever told me and the app wrote down. So what doesn’t belong in a chat doesn’t belong there either. The sticky note comes with me into every new chat.
How it works
The app’s memory
Some apps offer a “memory”. The app writes down notes from your conversations, such as your name, your job or that you have a dog, and adds them for the chatbot at the start of every new chat. That is why it “remembers” what you wrote a month ago.
The model has not changed or learned anything. It got a sticky note. That is convenient, but the same applies as in episode 4. The chatbot reads what is on the sticky note in every new conversation. So check in the settings what the app has written down, and delete what should not be there.
And one more thing that people confuse with learning. Some services use their users’ conversations to train new models. That is not memory either; your chatbot remembers nothing from it. It is a matter of the terms and the settings, and in most services you can switch it off. Company accounts often have different terms.
- Jana: So: the library, the notebook and the sticky note. Three different things.AI: And none of them is what you call memory. I don’t remember anything. I just read what is in front of me.
Three habits
- The next dayJana: Right, Martin, three things. First: one chat per task.Martin: I had an offer, a holiday in Croatia and a goulash recipe in one chat.
- Jana: Second: repeat the key facts. Who it’s for, what, and by when. Especially when the chat is long.Martin: So no more “like I told you at the start”.
- Jana: And third: paste only the part it needs. Not the whole annual report for the sake of one table.Martin: I put those forty pages in just to be safe.
How it works
Three habits that get around forgetting
The chatbot will not start remembering better. But you can write so that the important things are always in the notebook.
- One chat per task. A new thing, a new chat. The notebook stays clear, and nothing important gets pushed out.
- Repeat the key facts. Who it is for, what, in what tone, by when. In a long chat, more than once if needed. Whatever is newest in the notebook will not be the first to fall out.
- Paste only what is needed. A chapter instead of the whole report, a table instead of the whole attachment. It takes up fewer pages, and the answer is usually more accurate.
- Jana: And what will you remember from today?AI: As long as it fits in the notebook, everything. In a new chat, nothing, even though the app saves the conversation. But you will remember it, and that is what matters.
Quiz
- AI: Before you close this chat, five questions. You will remember them. I won’t.Jana: That is almost touching.
- Quiz
Question 1 of 5
Five questions, no marks.
What takes up space in a chatbot’s notebook?
Choose all the right answers.
What to take away
- The AI’s notebook, known as the context window, holds one conversation. Everything it knows about you right now is on its pages, including the texts you pasted.
- There is a limited number of pages. When they run out, the oldest lines fall out. How much fits depends on the app and the plan.
- A new chat starts empty. The “memory” of some apps is a sticky note that the app keeps about you. The chatbot does not learn from your chats. Its training ended before it was released.
- Three habits: one chat per task, repeat the key facts, paste only the part the AI really needs.
Next episode
#6 How to ask AI
The same task, five different prompts. Switch ingredients on and watch the answer change.
New episodes straight to your inbox
Every two weeks we send the new episodes, a short learning note and news from the world of AI. Free, and you can unsubscribe any time.