Why a smaller model can still win
Along the same lines, People like to compare models the way they compare GPUs or phone cameras. Bigger number, newer release, better result. That works until the model has to do actual work inside a real project, where half the job is remembering what the team already decided three weeks ago and not asking the same question for the fifth time.
A fresh chat starts blank. It does not know your codebase history, your naming conventions, the deployment constraints nobody wants to revisit, or the little rule you set after that one incident in staging. It doesn’t know that customer_id is the standard, not clientId. It doesn’t know the API wrapper you built around a flaky service. M. Because ops shuts down the database for maintenance then. A long-lived workspace, by contrast, has already absorbed those details. That memory changes the quality of the output in a very ordinary way: fewer false starts, fewer off-target suggestions, less back-and-forth.
A model without project memory is often just guessing in a nicer font.
That said, that’s why a slightly weaker model can beat a stronger one on day-to-day tasks. Folder structure, along with terminology and rough goals in its prompt memory. It can produce something more usable than a brilliant model that keeps treating every request like a cold start, if the smaller model already has the project’s prior decisions. The raw benchmark score matters less than people think once the work becomes repetitive and specific. Most of the frustration comes from forcing a smart system to rediscover things your team already knows.
This means this shows up everywhere. A model that remembers your preferred tone can draft copy in the right voice without a cleanup pass. A model that knows your backend stack can avoid suggesting libraries you never use. A model that’s the latest deployment notes won’t tell you to “just restart the service” when there’s a strict rollout window and a post-deploy validation step. None of that’s flashy. It is, however, the difference between a useful assistant and an expensive autocomplete with opinions.
The real advantage, then, isn’t the chat UI and not the model brand. It’s the durable AI project context around the work. Along with tools and model swaps, your team stops re-explaining the same project facts every time the window resets, when that context survives across sessions. That’s where context portability starts to matter more than whichever model happens to top the leaderboard this month.
The rest of this article breaks that problem into three practical questions: what context deserves to be kept, where it should live so it doesn’t rot inside one app, and how to move it cleanly between tools without turning every handoff into a copy-paste ritual. Once those pieces are in place, model choice becomes a lot less dramatic. The system remembers the project, and the model just does the reading.
What deserves to live in context?
Then once you stop treating every model swap like a clean slate, the next question gets more practical: what actually belongs in the portable part of your project memory, and what should be left behind in the chat scrollback where it belongs?
On top of that, the useful stuff is the stuff that keeps paying rent. Goals do. So do architecture decisions, naming conventions, environment assumptions, and the rules your team keeps rediscovering the hard way. A fresh model can guess at style from examples, but it can’t infer that your team always uses snake_case for database fields, that prod runs behind a strict egress policy, or that one API is approved while another is off-limits. But it can’t infer that your team always uses snake_case for database fields, that prod runs behind a strict egress policy, or that one API is approved while another is off-limits.
If a note won’t change the next decision, it probably doesn’t deserve a seat in shared context.
That simple filter helps separate durable project memory from disposable discussion. A long thread about whether to use Redis or PostgreSQL for a cache layer may be useful while the decision is open. The transcript matters less than the result: which system you, on second thought, picked, why you picked it, and what constraints led you there, once the choice is made. The same goes for style debates, and keep the convention, not the argument. Keep the spelling of a package name, not the three-page detour that got you there.
For most teams, the best home for that material is a small set of canonical docs rather than a pile of raw transcripts. A project overview gives the assistant the shape of the work. A decision log records what was chosen and why. Coding conventions pin down formatting, error handling, and naming. A glossary clears up terms that get used in-house but mean little outside the repo. That structure is boring in the best way. It makes model switching less annoying because the new model sees the same source of truth the old one saw.
Hard constraints deserve special treatment because they save the most time when they’re stated plainly. If the team’s approved APIs, write them down (and yes, that matters). Write those down too. Background workers, or outbound calls to a certain service, don’t make the next assistant rediscover that after writing a clever but unusable plan, if the system cannot depend on GPU inference, if there are systems limits. Team defaults belong here as well: logging format, testing framework, deployment target, regions you support, retry limits, whatever keeps the work grounded. The same goes for explicit “don’t do” rules. Those are often more useful than preferences. “Do not change the schema without a migration.” “Do not introduce a new queue unless the current one is exhausted.” “Don’t add a dependency just to save three lines.” That sort of thing saves real time.
Transient debate is different. So are stale hypotheses. A model can get distracted by a long argument that no longer matters, especially in an LLM workflow where context windows are precious and every extra paragraph competes with the actual task. If a theory was never tested, label it as a theory. If a workaround was temporary, say so. To some degree, if a note was written for one incident and doesn’t describe the system anymore, retire it. Otherwise, the next model may treat yesterday’s guess like today’s rule.
Still, the cleanest boundary is usually this: keep facts that affect future work, and drop text that only explains how you arrived there. That sounds almost too neat, but it saves a lot of mess.
If you want a concrete mental model, think for prompts and state. OpenAI’s conversation state guidance and function calling guide both point toward the same idea: the useful bits are the structured bits. Facts, constraints, and tool outputs survive handoffs far better than sprawling dialogue. Big difference. That matters whether you’re preserving project memory across agents or just trying to keep one assistant from forgetting your naming rules every third request.
So, before you copy a whole chat into the next tool, ask a blunt question: what would someone need to do the next task correctly? Keep that. Trim the rest.
Build a portable context layer, not a chat history
the next question is where that context should live, once you’ve decided what deserves to live in context. It looks like, my advice: keep it in files the rest of the project already understands. Markdown for human-readable notes. YAML for structured settings. JSON for data you want a tool to parse without guessing. If the memory only exists inside one chat product, it’ll age badly the first time you switch IDEs, replace your assistant, or need an AI handoff to a teammate’s workflow.
A portable setup usually looks boring, which is a compliment. You might keep context/project.md for the overview, context/decisions.yaml for architecture choices, and context/glossary.json for terms the model keeps mixing up. Put those files under version control with the codebase. Then a prompt can pull from them on demand instead of dragging an entire conversation along for the ride. That makes context management much less fragile. You get a single source of truth, and every tool that can read text can use it.
If the assistant can’t read it, search it, and version it, it isn’t memory. It’s just leftovers from a chat window.
The trick is to keep the bundle compact. You do not want a prompt stuffed with six weeks of brainstorming. Deliberate context package: project name, current goal, hard constraints, naming rules, open decisions, and maybe a few examples of the tone you want, you want a small. The rest should be retrieved only when needed. For instance, a task about API error handling should load error conventions and retry rules, not the notes about the marketing site or that one argument about folder names that nobody has thought about since Tuesday.
That retrieval step matters because most tasks are narrow. A good assistant should fetch the few notes that match the request, not shovel in every file the repo’s ever seen. OpenAI’s file search guide is a decent reference point for this pattern: store the source material separately, then let the model pull the relevant bits at query time. If you reuse the same prompt scaffolding often, prompt caching can also help reduce the cost of sending the same context block over and over.
Here’s a plain example in Python. It reads a Markdown context file, fills a prompt template, and gives you one place to swap in fresh project memory:
from pathlib import Path
context = Path("context/project.md").read_text(encoding="utf-8")
template = """You are helping with this project.
Project context:
<picture>
<source data-lazy="@srcset /assets/images/blog/post-1782862934/build-a-portable-context-layer-not-a-chat-history-320px.webp" media="(max-width: 320px)" type="image/webp">
<source data-lazy="@srcset /assets/images/blog/post-1782862934/build-a-portable-context-layer-not-a-chat-history-640px.webp" media="(max-width: 640px)" type="image/webp">
<source data-lazy="@srcset /assets/images/blog/post-1782862934/build-a-portable-context-layer-not-a-chat-history-1024px.webp" media="(max-width: 1024px)" type="image/webp">
<source data-lazy="@srcset /assets/images/blog/post-1782862934/build-a-portable-context-layer-not-a-chat-history.webp" type="image/webp">
<source data-lazy="@srcset /assets/images/blog/post-1782862934/build-a-portable-context-layer-not-a-chat-history-320px.jpg" media="(max-width: 320px)">
<source data-lazy="@srcset /assets/images/blog/post-1782862934/build-a-portable-context-layer-not-a-chat-history-640px.jpg" media="(max-width: 640px)">
<source data-lazy="@srcset /assets/images/blog/post-1782862934/build-a-portable-context-layer-not-a-chat-history-1024px.jpg" media="(max-width: 1024px)">
<source data-lazy="@srcset /assets/images/blog/post-1782862934/build-a-portable-context-layer-not-a-chat-history.jpg" media="(min-width: 1025px)">
<img
src="data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="
data-lazy="@src /assets/images/blog/post-1782862934/build-a-portable-context-layer-not-a-chat-history.jpg"
class="img-fluid rounded-3 w-100 my-5"
alt="Build a portable context layer, not a chat history"
>
</picture>
Task:
Write the release checklist for the next deploy.
"""
prompt = template.replace("", context)
print(prompt)
The Node version is just as plain:
```javascript import fs from “node:fs”;
const context = fs.readFileSync(“context/project.md”, “utf8”);
const template = `You are helping with this project.
Because of this, Project context:
Task: Write the release checklist for the next deploy. Md` with dated entries for things like API choices, data format rules, and deployment assumptions. Maybe a simple loader script that reads those files and injects the relevant parts into prompts. Nothing dramatic. Nothing that needs a committee. Just enough structure so the context is not trapped in whatever app happened to host the last conversation.
Once that habit is in place, the payoff starts stacking up. The model asks fewer dumb follow-up questions because it already knows the preferred style, along with the internal terminology and the awkward edge cases that came up three weeks ago. You stop repeating the same project summary every time you open a new tool. New teammates spend less time guessing which version of the truth is current. Cross-tool collaboration gets less clunky because everyone is reading from the same memory, even if they’re using different interfaces.
There’s also a quieter perk Teams make fewer accidental contradictions. One assistant suggests a rewrite, another agent generates a script, and both are working from the same project facts instead of two slightly different memories. Arguably, that matters more than people expect. A lot of friction in AI-assisted work comes from context drift, not model quality. The output looks fine until you notice that every tool has a different idea of what the project actually is.
So here’s a simple filter for choosing AI tools and workflows: if the context can’t move, the setup will age badly. In a way, if the memory can be exported, reloaded, and reused without much fuss, you’ve got something worth keeping. Start with a small, clean context file. Add a decision log, and make retrieval easy. Then let the tools compete on output, not on how well they trap your project history inside their walls.
