What are AI agents?
An AI agent is a system where an LLM decides its own steps and calls tools in a loop until a task is done. How that differs from a workflow, and when to use one.
An AI agent is a system in which a large language model (LLM) decides what to do next and uses tools to do it, repeating until the task is finished. Anthropic describes agents as systems where “LLMs dynamically direct their own processes and tool usage,” and as “typically just LLMs using tools based on environmental feedback in a loop” (Building effective agents, published 19 December 2024). IBM gives a similar definition: “a system that autonomously performs tasks by designing workflows with available tools” (IBM).
For the wider idea of systems that act with autonomy, see agentic AI.
What an agent is made of
Vendors name the parts differently, but the same pieces recur.
| Part | What it does | Where it is described |
|---|---|---|
| Model | Reads the task and the results so far, then decides the next step | Anthropic, IBM |
| Tools | Let the agent act: function calls, web search, APIs, other agents, MCP servers | IBM, OpenAI |
| Loop | Runs the model, executes the tool it chose, feeds the result back, and repeats | Anthropic, OpenAI (orchestration) |
| Plan | Breaks the goal into “specific tasks and subtasks” | IBM |
| Memory and state | Keeps past interactions or saved progress across steps | IBM, OpenAI |
OpenAI also lists handoffs between agents and an optional sandbox for running code as parts of its agent setup (OpenAI). Not every agent has a planner or memory. The model, the tools and the loop are the minimum.
Agent or workflow
Anthropic separates two kinds of agentic system. In a workflow, “LLMs and tools are orchestrated through predefined code paths.” In an agent, the LLM chooses its own path and tools (Anthropic). By this definition, many products sold as agents are workflows. Anthropic notes that workflows “offer predictability and consistency for well-defined tasks.”
| Single LLM call | Workflow | Agent | |
|---|---|---|---|
| Who picks the steps | You, in one prompt | Your code | The model |
| Number of steps | One | Fixed in advance | Not known in advance |
| Fits | One well-scoped task | Well-defined tasks | “Open-ended problems where it’s difficult or impossible to predict the required number of steps” |
| Cost and latency | Least of the three | Between the two | “Agentic systems often trade latency and cost for better task performance” |
| Risk Anthropic names | None given | None given | “The potential for compounding errors” |
The quotes in the Agent column come from Anthropic. The other cells follow from the definitions above.
The practical rule that follows is to use the simplest option that works. Move from a single call to a workflow, or from a workflow to an agent, only when the task needs it.
Types of agents in older textbooks
IBM lists five classic types, from simplest to most advanced: simple reflex, model-based reflex, goal-based, utility-based and learning agents (IBM). They come from classic AI research, before LLMs. Of the five, goal-based agents are the closest match to the definition above: they get a goal and pick actions toward it.
Why agents are hard to debug
An agent’s path is different on every run, so you cannot read the code to see what happened. Errors compound because each step’s output becomes the next step’s input. The usual way to inspect a run is a trace: one record per task, with each model call and tool call nested inside it. Agents can loop, call the wrong tool or pass bad arguments, and a trace shows which step it happened at. See agent observability for what to record, and LLM evaluation for scoring the results.
Frameworks
Anthropic describes an agent as an LLM using tools in a loop, so you can write that loop yourself. Agent frameworks package it. LangGraph and Pydantic AI are covered on their own pages.
Tracing an agent in HoneyHive
HoneyHive records each run as a session. Every model call, tool call and group of steps is an event nested under it, with the types model (“An LLM API request”), tool (“An external service or function call”) and chain (“A logical grouping of child events”) (HoneyHive tracing concepts). That maps onto the loop above: each pass through the loop shows up as a model event followed by a tool event.
FAQ
What is the difference between an AI agent and a chatbot?
A chatbot replies to each message. An agent takes a task and works through it, choosing tools and next steps itself, until it decides the task is done. A chatbot can be built as an agent by giving the model tools and a loop, but a plain reply-only chatbot has neither.
What is the difference between an AI agent and a workflow?
In a workflow, code fixes the path and the LLM fills in steps. In an agent, the LLM chooses the path and the tools. Anthropic draws this line in Building effective agents.
Are AI agents always better than a single LLM call?
No. Anthropic notes that agentic systems often trade latency and cost for better task performance, and that autonomy brings higher costs and the potential for compounding errors. For a well-defined task, a fixed workflow or one call is often enough.
How do you see what an AI agent did?
Record each model call and tool call as a span in one trace per task, then read the trace step by step. See agent observability.