There's a question I've spent far longer on than I expected while building DoDone: how to carry the context of a conversation forward in the chat window. When you build an AI agent product, it's easy to think about this simply at first. Can't you just put every previous message into the context?
Model context windows keep growing. Hundreds of thousands of tokens, and now models that support a million. So it looks like the best approach is to feed the model as much of the user's conversation as possible. But the conclusion I reached while building DoDone was a little different.
An AI's memory is less about how much you can fit in, and more about deciding what it needs to remember right now.
The most intuitive way to handle long conversations
Agents like Claude Code and Hermes use a technique called compaction to keep long tasks going. Anthropic also describes compaction as a core method for managing context in long-running agents. When a conversation grows, instead of keeping every old message, you summarize what matters and build a fresh context alongside the recent turns. Claude Code likewise compresses old messages and work while preserving key design decisions, open problems, and implementation details.
Hermes's implementation is more concrete. It doesn't compact based on the number of messages but on a token budget. When the actual prompt tokens used in the current API call reach a set fraction of the context window, compaction kicks in automatically. The default threshold is 50%, and for models with windows smaller than 512K it's raised to at least 75% to avoid compacting too early. Some recent messages are kept verbatim while the middle stretch is summarized.
It's a very reasonable structure. It fits especially well when a single goal runs for a long time — one coding task, one research thread. But DoDone had a slightly different problem.
Why DoDone doesn't just keep feeding the whole conversation
DoDone is a virtual office where AI colleagues work. It isn't only about one AI and one topic discussed deeply for hours. A user might talk to the marketing colleague, then hand work to the developer, hold a meeting in a project channel, then pass a follow-up back to a specific colleague — and come looking for that colleague again a day later.
In other words, chat in DoDone isn't just a conversation window. It's the interface where work happens. That difference changed the context design. If you keep passing every past message to the model as the conversation grows, you can remember a lot — but three problems appear at the same time.
The first is cost. An LLM re-reads the context on every request. The longer the conversation, the more you keep re-sending the same old messages. Especially on models with very large windows, if compaction happens late, the cost of a whole session can rise sharply. Even in Hermes, there's discussion that with a 1M model and the default 50% threshold, the first compaction might not fire until 500K tokens.
The second is speed. More input tokens mean more information for the model to process. There's no need to make it re-read weeks-old conversations that have nothing to do with the current question.
But the one I weighed most heavily was the third: context pollution.
More context isn't always better
In explaining context engineering, Anthropic points out that relevance and context pollution remain problems even with long contexts. That's an important point. A million-token window doesn't mean filling it with a million tokens is the best state to be in.
Say a user had this series of talks with the marketing colleague. Last week, branding for a new service. A few days ago, Instagram ad copy. Yesterday, the product's pricing policy. And today, they ask for a press release. Technically you can pass all of it to the model. But if today's job is a press release, does dragging in the verbatim back-and-forth of editing Instagram copy from weeks ago actually help? It just creates room for information unrelated to the current task to intrude on the model's judgment.
So from the start, DoDone cared less about 'how much conversation to include' and more about 'how recent a conversation to keep in full.'
DoDone's approach — recent in sharp focus, older compressed
Today DoDone keeps the most recent 8 tasks as detailed context. A colleague conversation uses a detailed-context budget of about 20K characters, and the master channel about 16K. Up to the last 8 tasks, details are kept as intact as possible; anything older is compressed into a rolling summary that carries the earlier context forward.
Simplified, the structure is this: past tasks 1–12 become a rolling summary, recent tasks 13–20 stay as detailed context, and what you're doing now is the current mission.
The key is that old conversations aren't deleted — their resolution is lowered. Recent turns are remembered in high resolution, older ones in low resolution. I think this is quite close to how human memory works. We don't remember a conversation from a week ago sentence by sentence. But the important context stays: 'we set the price at $29 on that project,' 'we chose solopreneurs as the target customer,' 'the next task was building the landing page.' I judged that long AI conversations feel more natural with the same structure.
But summaries have limits too
Of course, a rolling summary isn't a cure-all. A summary is compression, and the more you compress, the more detail you inevitably lose. And once 20, 30, or 50 tasks pile into a single conversation, another problem appears: it becomes unclear whether the user is even still doing the same thing.
You might start by discussing marketing strategy, build a homepage in the middle, talk about hiring after that, and then start creating content again. Technically it can all stay connected in one session. But 'technically possible' and 'good UX' are different things. So DoDone recently added a small, fun mechanism.
The AI is the first to say, 'Try starting a new conversation'
When missions in the current session reach 20, DoDone tells the user first. That's the point where the detailed recent 8 plus roughly 12 folded into the rolling summary have accumulated. At that moment, this message appears:
“💡 This conversation has gotten pretty long. If the topic has changed, I'd suggest starting a new session — responses get more accurate and AI costs drop. If you keep going, the earlier flow is still kept as a summary.”
The important thing is that it never forces the session to end. If you're continuing the same project, just keep going. But if the topic has already shifted, starting a new session is far better — and even then, the earlier flow you need carries over through the summarized context. In the end, the choice stays with the user.
Where to surface that notice mattered too
I didn't settle for triggering the long-conversation notice by message count alone. DoDone checks it in two places.
The first is the moment a task instruction comes in. Whether it arrives through a 1:1 with a colleague, the master channel, the all-hands channel, a meeting follow-up, or a project, the session length is checked asynchronously right after the instruction is received. If the condition is met, the notice appears naturally beneath the instruction the user just sent.
The second is the moment the user enters a channel. When you click a colleague in the office to open a chat, or step into a project channel, the current session's state is checked. If it's already long enough, you can see the notice before even sending a new message. And within the same session, you're only told once; starting a new session makes it eligible again. It's a very small piece of UX, but in an agent product, this kind of thing matters quite a lot.
In an AI agent, memory and context are not the same
One thing keeps getting clearer as I build DoDone: memory and context should not be treated as the same thing.
Memory is what a colleague needs to know over the long term. User preferences, company information, ways of working, recurring rules, important facts decided on past projects. This information may be needed days or months later. Context is different — it's closer to the working memory needed to do this task right now. The immediately preceding request, the file being worked on, what was just decided, the problem to solve now. Try to solve both by cramming them into the chat history and the system gets heavy fast.
So I think a good agent system ends up with several layers of memory: the current mission (what's being done now), recent context (the last few detailed interactions), a rolling summary (the compressed flow of past work), and long-term memory (facts and rules that must persist regardless of the session). The model's context window is just the space where the ones you need are assembled in that moment.
The bigger the context window, the more context engineering matters
There's an irony here. Back when context windows were small, developers had no choice but to cut information down. As windows grew to 128K, 200K, 1M, we can now fit a lot in. And that created a new problem: you have to decide what not to include.
A good AI agent may not be the one that remembers everything. What's needed now is an AI that remembers the present sharply, compresses the old appropriately, moves the important into long-term memory, and creates a new workspace when the topic changes. In the end, the work you do more and more while building an agent isn't prompt engineering — it's context engineering.
The core question is shifting too. It used to be 'what should we tell the model?' Now it's closer to 'in this moment, how much should the model actually know?'
This is exactly what I've spent the longest on while building DoDone. Not making the AI remember more, but making it remember the right things at the right moment. I believe the difference that decides the quality of AI agents from here on will be made right there.
— Seungwon Go, the solopreneur building DoDone