What is human-in-the-loop AI?

Human-in-the-loop (HITL) AI is a design where a person takes part in an automated system’s workflow, by labeling data, reviewing outputs or approving actions. IBM defines it as a setup where “a human actively participates in the operation, supervision or decision-making of an automated system”. For AI agents, it most often means the run pauses before a risky step and waits for a person to approve, edit or reject it.

Where people sit in the loop

IBM lists several uses:

• Supervised learning, where humans label examples such as “spam” or “not spam”.
• Reinforcement learning from human feedback (RLHF), where a reward model trained on human feedback guides the AI.
• Active learning, where the model flags uncertain predictions and humans label only those.
• Overrides in high-stakes settings such as healthcare, finance and hiring.

The first three improve a model. The last is runtime oversight, and it is what agent frameworks mean by the term.

Why agents need it

Anthropic says agents “can then pause for human feedback at checkpoints or when encountering blockers”, and that “human review remains crucial for ensuring solutions align with broader system requirements.” It also recommends “extensive testing in sandboxed environments, along with the appropriate guardrails.”

On security, OWASP defines excessive agency as “the vulnerability that enables damaging actions to be performed in response to unexpected, ambiguous or manipulated outputs.” One of its mitigations is to “utilise human-in-the-loop control to require a human to approve high-impact actions before they are taken.” The same page says the check can sit in a downstream system or in the extension itself. See also prompt injection.

For regulated systems, Article 14 of the EU AI Act requires human oversight of high-risk AI that aims “to prevent or minimise the risks to health, safety or fundamental rights”. Measures must be “commensurate with the risks, level of autonomy and context of use”. Overseers must be able to “disregard, override or reverse the output” and to interrupt the system with “a ‘stop’ button or a similar procedure”. The article also names automation bias: overseers must stay aware of “automatically relying or over-relying on the output”.

How frameworks implement it

All three below pause a run before a tool executes. They differ in what the reviewer can do and how the run resumes.

LangChain’s reviewer can edit a call’s arguments before it runs, as can a Claude Agent SDK callback. LangChain’s respond decision returns a human reply as the tool result, and the docs warn: “Do not use respond to deny side-effecting tools.” Its batched mode (the default when read) raises one interrupt per model turn covering every gated call, and interrupt_mode="per_call" (langchain 1.4.6 or later) raises one each. The docs recommend per-call mode for new agents and plan to make it the default in the next major release. LangGraph users get the same pause through LangGraph checkpointing, and the OpenAI SDK is covered in OpenAI Agents SDK.

Details that cause bugs

• Approval is not exactly-once. The OpenAI docs say “Approval permits a tool call to execute; it does not guarantee exactly-once execution”, and recommend idempotency keys for consequential side effects.
• Authenticate the reviewer. In the OpenAI Agents SDK, RunState.from_json() does not authenticate the snapshot or the reviewer, so your server must check who is approving and validate decisions against pending items it stored itself.
• Auto-approval skips your callback. In the Claude Agent SDK, a call approved by an allow rule or by acceptEdits or bypassPermissions mode never reaches canUseTool. For a check on every call, the docs point to a PreToolUse hook.
• Modes change who decides. The SDK’s plan mode sends file edits to your callback, dontAsk denies anything that would prompt, and auto has a classifier model review actions instead of a person.

Deciding what to gate

Gate by consequence, not by tool count. OWASP’s wording is “high-impact actions”, and the EU AI Act asks for measures matched to risk and autonomy. In practice that means requiring approval for tools that send messages, move money, delete data or publish, and letting read-only tools run. LangChain’s interrupt_on lets you set True, False or a config per tool, and its when predicate can gate on a call’s arguments.

Constant prompts invite rubber-stamping, which is the automation bias Article 14 warns about. Keep the gated set small enough that reviewers read each request.

Measuring the human step

Review adds latency and cost, so check that it catches real problems. A trace of each run shows where the agent paused, what the reviewer saw and what happened next; see agent observability. Human judgments can also feed evaluation, for example to check an automated evaluator against expert labels.

HoneyHive’s docs cover annotation queues for human review of agent outputs and expert labels that become ground truth for evaluators, and human evaluators for scores, notes and labels on traces. Approval gates inside an agent run are handled by your framework as above.

FAQ

What does human-in-the-loop mean in AI?

IBM defines it as a setup where "a human actively participates in the operation, supervision or decision-making of an automated system" (IBM). In training that means labeling and feedback. In agents it usually means approving actions.

How do I add human approval to an AI agent?

Most frameworks pause the run before a tool call and wait for a decision. LangChain uses HumanInTheLoopMiddleware with interrupt_on, and the OpenAI Agents SDK uses needs_approval. The Claude Agent SDK calls your canUseTool callback.

Does LangChain human-in-the-loop need a checkpointer?

Yes. The docs say "Human-in-the-loop requires checkpointing to handle interrupts." They suggest a persistent checkpointer such as AsyncPostgresSaver for production and InMemorySaver for testing (LangChain docs).

Does human oversight stop prompt injection?

It limits the damage. OWASP lists human approval of high-impact actions as a mitigation for excessive agency (OWASP LLM06:2025). It does not stop the model from being manipulated, so pair it with guardrails.