Context engineering: the skill that replaced prompt engineering

Prompt engineering as a discipline was largely a response to models that were hard to steer. You needed the magic phrasing, the roleplay framing, the threat of consequences, the "think step by step."

Most of that is gone. Current models follow instructions closely and literally — closely enough that the aggressive phrasing which used to be necessary is now actively harmful, producing over-triggering and over-caution. What replaced it is less glamorous and far more consequential: deciding what information the model sees, in what order, and what you leave out.

That's context engineering, and it's where production quality is actually won.

The window is a budget, not a container

Large context windows created a bad instinct: if it fits, send it.

It shouldn't. Two reasons.

Attention isn't uniform. Models attend unevenly across a long context. Information in the middle of a large window is measurably less influential than information near the start or end. Padding the prompt doesn't add signal — it dilutes it.

Every irrelevant token is a distractor. Ten passages where three are relevant is a harder problem than three passages that are all relevant. You're asking the model to do retrieval and the task, when you should have done the retrieval.

Treat the window like a budget with a real cost. Every token has to earn inclusion.

The anatomy of a production context

A well-built context has five distinct layers, and the ordering matters — both for attention and for caching.

1. System — identity, constraints, and format. Stable, frozen, cacheable. What the agent is, what it must never do, what shape the output takes. This should be identical across every request in the deployment. If it contains a date, a user name, or a mode flag, you have both a caching bug and a design smell.

2. Tools — the action surface. Renders immediately after system. Definitions must be deterministically serialised and stable across the conversation. Every tool description says when to call it, not merely what it does.

3. Retrieved knowledge — the facts for this task. Policy passages, documentation, prior similar cases. Filtered by metadata, ranked, reranked, and cut hard. Five good passages, not twenty adequate ones.

4. State — the live specifics. The actual order record. The customer's tier. Today's date. Inventory. This is the layer that separates a grounded answer from a plausible one, and it's the layer teams most often skip in favour of hoping the model infers it.

5. History — the conversation, pruned. Recent turns in full; older turns summarised or cleared. Tool results from twenty steps ago are rarely load-bearing and always expensive.

Stable content first, volatile content last. That ordering is what makes caching work, and it happens to match how attention behaves.

Retrieve state, don't describe it

The most common context failure isn't too much information. It's the wrong kind.

Teams write elaborate system prompts describing the returns policy in prose, then wonder why the agent applies it inconsistently. The policy has exceptions, market variations, product-category rules and date boundaries — expressed as paragraphs, they're something the model has to interpret every single time, differently each time.

Expressed as a tool that returns a decision — check_return_eligibility(order_id) → {eligible: true, reason: "within 30-day window", refund_method: "original"} — it's a fact. Facts don't drift.

Anything computable should be computed. Dates, eligibility, totals, tiers, thresholds. Put the result in context, not the rules for deriving it.

Structure it so it's parseable

Models handle structured context better than prose walls. This isn't superstition — it's the same reason humans do.

• Delimit sections clearly. XML-style tags, Markdown headings, consistent labels. Whatever you pick, be consistent across every request.
• Label retrieved passages with their source and date. [Returns Policy · EU · v2026.1 · effective 2026-01-15]. This enables citation, makes recency legible, and lets you validate claims afterwards.
• Put the instruction near the end, after the context it operates on. The model reads the material, then the task.
• Use the same structure every time. Consistency is itself a signal — it means the model isn't spending effort figuring out the shape of each new prompt.

Managing long-running agents

Agentic sessions accumulate. Twenty tool calls in, the window is mostly historical tool output that no longer matters. Three techniques, and they're complementary rather than alternatives:

Pruning. Clear old tool results and stale reasoning outright. Most providers support this server-side now. The conversation structure stays intact; the dead weight goes. Cheapest option and the right default.

Compaction. Summarise earlier history into a compact form when you approach the window limit. Preserves the thread of what happened, at the cost of detail. Note that if your provider returns compaction state, you have to pass it back — dropping it silently loses the summary.

Persistent memory. A file the agent reads and writes across sessions. This is underused and surprisingly effective: agents given a scratchpad measurably improve at long-horizon work. Give it structure — one lesson per file, a one-line summary at the top, an explicit instruction to consult it before starting, and instructions to update rather than duplicate.

What to leave out

Easier to get wrong than what to include.

Instructions for situations that can't arise. Every conditional you add is a branch the model evaluates. If the agent only ever handles post-purchase queries, it doesn't need pre-sales guidance.

Aggressive emphasis. CRITICAL:, YOU MUST, ALWAYS, If in doubt, use the tool. This language was written to overcome older models' reluctance. Current models follow it literally, which means over-triggering — tools called when they shouldn't be, escalations that weren't needed, hedging on questions that had clear answers. If a tool is over-firing, the fix is almost always to soften the language, not to add another guardrail.

Verification scaffolding. "Double-check your answer before responding." "Include a final verification step." Recent frontier models verify their own work without being told, and instructing them to do it produces redundant loops and wasted tokens. This inverts a long-standing best practice, which is exactly why it catches people — a prompt library that applies "ask it to self-check" uniformly needs a carve-out.

Few-shot examples you no longer need. Examples were essential when models were weaker. On current models they often narrow behaviour — the model pattern-matches the examples rather than reasoning about the case. Test with them removed; you may be paying tokens for a constraint you don't want.

Redundant restatement. Saying the same constraint in the system prompt, the tool description, and the user message doesn't triple its weight. It triples the cost and adds ambiguity when the three phrasings drift apart.

The debugging move

When an agent behaves wrongly, the instinct is to edit the prompt. The better first move is to look at exactly what was in the context on that request.

Log the fully rendered context. Not the template — the final bytes. Then read it as if you were the model.

You will find, more often than not:

• The relevant passage wasn't retrieved at all
• It was retrieved but ranked ninth, buried in the middle
• A stale document contradicted the current one and both were present
• The state layer was missing entirely and the model inferred from general knowledge
• Two instructions in different layers said different things

None of those are prompt problems. All of them are commonly "fixed" by rewriting the prompt, which produces a change that appears to help on the case you were looking at and helps nothing systematically.

The one-line version

Prompt engineering asks: how do I phrase this? Context engineering asks: what does the model need to see, and what is it seeing that it shouldn't?

The second question is where the wins are.

---

We instrument context before we tune prompts — rendered logs, retrieval scores, per-layer attribution. Book an audit call.