What an AI agent actually is (and the four parts that make one work)

The word agent has been stretched to cover everything from a chatbot with a system prompt to a fully autonomous engineering system. That vagueness costs money: teams buy an "agent platform" expecting autonomy and get a form with a language model behind it, or they build an autonomous loop for a task that needed one API call.

Here is the definition we use in production, and the four components you actually have to build.

The definition

An AI agent is a system where a language model decides what to do next, and those decisions have effects in the world.

Two clauses, both load-bearing.

The model decides — that separates an agent from a workflow. In a workflow, you wrote the control flow: step one calls the classifier, step two branches on the label, step three sends the email. The model fills in blanks. In an agent, the model chooses the sequence. You didn't know in advance whether it would search the knowledge base first or check the order status first.

Decisions have effects — that separates an agent from a chatbot. A chatbot produces text a human reads and acts on. An agent issues the refund.

If neither clause is true, you don't have an agent, and that is usually good news. Agents cost more, run slower, and fail in stranger ways than the alternatives.

Should this even be an agent?

Before the architecture, the triage. We check four things:

• Complexity. Is the task genuinely hard to specify in advance? "Extract the invoice total from this PDF" is not. "Investigate why this customer is unhappy and resolve it" is.
• Value. Does the outcome justify higher latency and higher token spend? An agent turn can cost 20× a single classification call.
• Viability. Is the model actually good at this class of task today? Be honest — measure, don't assume.
• Cost of error. Can a mistake be caught and reversed? Agents that write to a staging branch are fine. Agents that wire money need a different design.

If any answer is no, drop a tier. Most work that gets branded as agentic is better served by a workflow: you own the control flow, the model handles the language-shaped steps, and you can debug it.

The four components

1. The model

The reasoning core. Two decisions matter here and they're often confused.

Which model. Route per step rather than per system. A fast, cheap model classifies intent; a frontier model handles anything customer-facing or judgement-heavy. This is not a compromise — it's usually more accurate and cheaper, because the small model runs the same narrow task a thousand times a day and you can evaluate it properly.

How much reasoning. Modern models expose a reasoning-effort control. Higher effort means more deliberation, more tool calls, more tokens; lower effort means faster, terser, more literal execution. This is a dial to sweep on your own evaluation set, not a value to copy from a blog post. In our deployments the sweet spot for routine operational work is usually one notch below the default, and the highest settings are reserved for long-horizon autonomous runs.

2. Tools

Tools are the agent's hands. The design question isn't which tools exist — it's at what granularity you expose them.

A single bash tool gives the model enormous reach. It also gives your harness a single opaque string for every possible action, which means you cannot gate, audit, render, or parallelise any of it.

Promote an action to its own dedicated tool when you need to:

• Gate it. Hard-to-reverse actions — sending an email, deleting a record, issuing a refund — should be individually approvable. send_refund(order_id, amount) can require human confirmation. bash -c "curl -X POST ..." cannot.
• Enforce an invariant. A dedicated edit_file tool can reject a write if the file changed since the agent last read it. A shell command can't.
• Render it. Some actions deserve custom UI. An ask_user tool can open a modal with options.
• Parallelise it. Your harness can mark a read-only search tool as safe to run concurrently. It can't tell a safe grep from an unsafe git push when both arrive as shell strings.

Rule of thumb: start broad, promote as you learn. And be prescriptive in tool descriptions — say when to call the tool, not just what it does. "Call this when the customer asks about delivery timing or a missing parcel" outperforms "Gets order tracking information" by a wide margin, because the description is the only thing the model reads when deciding.

3. Context

The agent's working memory. This is where most production agents actually fail, and it gets its own article, but the short version:

• Retrieval puts the right facts in front of the model at the right moment. An agent that answers from model memory rather than from your live order data will be confidently wrong.
• Compaction and pruning keep long runs from drowning. As a session accumulates tool results, old outputs stop earning their place in the window. Either summarise the history or clear stale tool results outright.
• Persistent memory carries learnings between sessions. Even a plain Markdown file the agent reads and writes measurably improves long-running agents — one lesson per file, a one-line summary at the top, and explicit instructions to consult it.

4. The control loop

The part you own. At minimum:

while not done:
    response = model(context, tools)
    if response.wants_tool:
        result = execute(response.tool_call)   # your gate lives here
        context.append(result)
    else:
        done = True

Every SDK ships a version of this. Use theirs — the per-turn hooks give you approval gates, error interception, retries and result modification without hand-writing the loop. Write your own only when you need control the harness genuinely doesn't expose.

What matters is what you put inside it:

• A stopping condition that isn't just "the model stopped." Cap iterations. Cap total tokens for the task. Detect loops where the agent calls the same tool with the same arguments three times.
• An escalation path. A confidence threshold that hands off to a human with a written summary is worth more than any amount of prompt tuning. The agent that knows what it doesn't know is the one you can deploy.
• A complete audit trail. Every tool call, every input, every result, every decision. You will need it — for debugging, for compliance, and for the eval set you build from real failures.

What this looks like in practice

A support agent handling "where is my order":

1. Model — a mid-tier model at low reasoning effort. The task is narrow and the eval set is large.
2. Tools — lookup_order(email_or_number), get_tracking(order_id), escalate_to_human(summary, reason). Three tools, each with a prescriptive description. No shell.
3. Context — the order record, the carrier's live tracking response, the brand's shipping policy for that market, and the last three messages of the thread. Nothing else.
4. Loop — maximum four iterations. If tracking returns nothing, or the customer used any language matching the distress patterns in our escalation set, hand off immediately with a summary.

That's it. It resolves the overwhelming majority of a real ticket queue, and every part of it is inspectable. It is not impressive on a demo stage. It works on a Tuesday.

The failure mode to avoid

The most common mistake we see is building the loop first. Someone wires up an autonomous agent, gives it broad tools, writes an ambitious system prompt, and then spends three months tuning prose to fix behaviour that a dedicated tool and a decision boundary would have fixed in a day.

Build the tools first. Build the retrieval first. Get the context right. The loop is the last and smallest part — and if you've done the other three well, it turns out you often didn't need an agent at all.

---

AIMAGENTIC builds production AI agents for support, content, retention and reporting — inside your stack, with your keys. If you want the ranked map of what's automatable in your operation, book an audit call.