“AI generated outputs shall be subject to human review.” 

That sentence, in one form or another, appears in AI policies, vendor responses, risk assessments, governance presentations, and contracts. It sounds responsible, and it may be completely accurate. Somewhere in the process, a person may indeed review something produced by the system.

But the sentence does not tell us who reviews the output, what the reviewer sees, how much time the reviewer has, whether the reviewer understands the system’s limitations, or whether the reviewer can reject the output without penalty. We routinely ask whether an AI system is fit for purpose. We should also ask whether the human reviewer is fit for the oversight role assigned.

Those details determine whether “human review” is an operating control or merely a description of one. Many organizations are developing increasingly sophisticated-looking AI governance programs, and much of the initial work necessarily occurs in documents. That work matters. Policies establish acceptable uses. Risk frameworks classify systems. Contracts allocate obligations. Assessments identify reviewers, approval requirements, escalation paths, testing procedures, and monitoring responsibilities. Documentation creates authority, assigns responsibility, and gives organizations a common account of how AI is supposed to be governed.

But an accurate statement of intended governance is not itself an operating constraint. A policy may require approval while the workflow effectively makes approval automatic. A committee may wield formal authority over high-risk AI deployments while learning about them only after implementation. A contract may prohibit a vendor from using customer data for model training while leaving unresolved which data and uses the restriction actually covers. A system may provide a right to contest its output while giving the affected person no useful explanation, evidence, or path to someone authorized to reconsider the result.

In each case, the commitment may be real and the control may be incomplete. This is not a new problem, and it is not unique to AI. Auditors have long distinguished a control’s design, implementation, and operating effectiveness, and have separately examined the evidence showing whether anything actually happened. AI governance does not replace that analysis. It applies it to a faster moving and less transparent environment, making the gaps easier to hide. 

Three Different Things Called a “Control” 

AI governance discussions often use the word “control” to describe three different things: the stated control, the mechanism that makes it operate, and the evidence that it operated. Treating those layers as collapsible or interchangeable is one reason AI governance can appear more mature on paper than it is in operation. 

The Stated Control

The stated control is what a given policy, contract, assessment, presentation, or governance framework says should happen. Examples include:

• High-risk AI uses require approval;
• AI-generated work must receive human review;
• Customer data will not be used for model training; and
• AI agents may act only within authorized parameters.

A stated control establishes an expectation against a seemingly objective standard. It may also allocate responsibility, define acceptable conduct, support accountability, and create an enforceable obligation. That makes it important. But it does not answer the operational question of whether the organization’s systems, workflows, and processes allow the expectation to be carried out and verified. 

The Operating Control

The operating control is the mechanism through which the stated protection is implemented in the actual environment. For human review, that mechanism may include the reviewer’s training, access to underlying information, time to conduct the review, ability to understand the system’s output, authority to reject the recommendation, and protection from incentives that quietly (and, sometimes, not so quietly) reward speed or assent. For high-risk approvals, it may include an intake process capable of identifying AI embedded in a larger product, criteria for classifying the use, an approval gate that occurs before deployment, and a process for reevaluating the system when its model, data, purpose, or level of autonomy changes. The difference is basic but frequently obscured: the stated control expresses an expectation. The operating mechanism determines whether the environment prevents inconsistent action, detects it, or responds when it occurs. 

The Evidence of Operation

The third layer is the record showing that the mechanism operated in a particular case or over a relevant period. Evidence may include:

• Approvals and overrides;
• Testing and monitoring results;
• System configurations and permission records;
• Version histories;
• Exceptions and escalation decisions; and
• Remediation records and evidence showing what a reviewer considered.

These records are not the control itself. A highly documented process can still be ineffective. But without evidence, an organization may be unable to determine whether the control operated, whether it operated consistently, or whether it failed in precisely the cases that mattered. Organizations sometimes move too quickly among these layers. The policy states the control, so the control is treated as present. Someone is assigned responsibility, so the control is treated as implemented. A box is checked, so implementation is treated as proved. The box rarely objects. 

What Makes an AI Control Operational? 

Drawing on familiar control-design principles, organizations can test an asserted AI control through eight operational elements: trigger, actor, information, authority, timing, evidence, response, and feedback. Not every control will require the same level of formality. But consequential AI uses should be traceable through each element: 

1. Trigger 

What event causes the control to operate? If a policy requires “additional review for high-risk AI,” how does the organization identify the relevant use — and does a material change in the model, data, use case, or autonomy level trigger reevaluation or some other requirement? A control without a reliable trigger operates only for the systems the organization successfully identifies, which is a more limited achievement than policy language may suggest. 

2. Actor 

Who or what performs the control? “The organization will review” is not an operating assignment; the organization must identify the function, role, or person responsible and determine whether multiple participants have related obligations. This becomes more important with agentic AI, where broad permissions can create exposure faster than anyone reviews them. A workflow may involve a user who initiates the task, an agent that plans and executes steps, a platform that controls permissions, a vendor that changes the underlying model, and a reviewer who sees only the final result. Assigning “human oversight” somewhere within that chain does not resolve which human oversees which action. 

3. Information 

What information does the actor receive? A reviewer may not be able to evaluate an output meaningfully if the interface supplies only the result and an unexplained or uncalibrated confidence score. Depending on the context, review may require access to the underlying inputs, relevant source materials, known model limitations, prior decisions, uncertainty indicators, applicable criteria, and records of any tools, data sources, or external actions used by the system. An approval process can preserve formal authority while withholding the information required to exercise it. 

4. Authority 

Authority should be understood practically, not formally. What practical authority, institutional support, and independence does the actor have? Can the reviewer reject the recommendation, stop the process, require additional information, modify the result, revoke the agent’s authority, or escalate the matter to someone who can?

A reviewer who can technically reject an output but is expected to clear several hundred items before the end of the day occupies a different control environment from a reviewer given the time and institutional support to disagree. The control environment includes not only permissions and escalation rights, but also the incentives that determine whether those rights are ever exercised. 

5. Timing 

Advance approval may prevent an action. Monitoring may detect an inappropriate action. Suspension, notice, rollback, or remediation may correct or contain its effects. Calling all three “human review” obscures what risk the mechanism can actually reduce. Review after an AI-generated communication has been sent, a transaction executed, or an agent has modified a production system may support detection and accountability, but it is not the same control as advance approval. The distinction sharpens as systems move from generating content to taking action. A person who approves the objective at the beginning of a workflow may not have approved the data sources consulted, tools invoked, permissions used, communications sent, records modified, or transactions executed along the way. For agentic systems, timing is not only about whether review occurs before or after an output. It is about where human judgment sits in relation to planning, tool use, execution, escalation, and termination. 

6. Evidence 

What record establishes that the control operated? An approval record may show that someone “approved” without showing what information was available, whether the reviewer conducted an independent evaluation, or why the recommendation was accepted. The necessary evidence should be proportionate to the risk. Not every AI-assisted email requires an adjudicative record. Consequential uses, however, may require more than proof that a person’s credentials appeared at the expected point in the workflow. Evidence from a single transaction may establish that the mechanism operated once. Assessing operating effectiveness ordinarily requires asking whether the control functioned consistently across the relevant population and period, including exceptions and edge cases.

7. Response 

What happens when the control identifies a problem — or when the control itself fails? A monitoring process that produces reports nobody reviews is a reporting function, not a complete control, and an escalation path that ends with a person who lacks authority to suspend the system describes the route by which the problem becomes stationary. Organizations should define who investigates, who may pause the use, how affected decisions are identified, and when legal, compliance, security, or executive personnel must become involved. For technology-mediated controls, implementation often resides in configuration rather than prose: permissions, default settings, workflow gates, logging, retention, tool access, and exception handling may determine whether the stated control can operate at all.

8. Feedback 

Does the failure change future behavior? A mature control creates a path for updating the system, configuration, permissions, training, documentation, risk classification, contractual protections, or deployment decision. This matters for AI because risk migrates after implementation: data changes, models are updated, users find new purposes, and vendors add functionality. An exception that produces no change in future practice becomes an institutional memory only in the ceremonial sense. 

Human Review Is Not a Single Control 

“Human in the loop” remains useful shorthand, but it can describe materially different arrangements. These arrangements differ not only in strength, but in kind. A person who independently evaluates underlying evidence performs a different control function from a person who reviews an AI recommendation, who performs a different function from a person who sees only system-flagged exceptions, who performs a different function from a person who merely appears in the workflow. All involve a human. None provide equivalent oversight. 

The distinction matters beyond policy design. For high-risk systems subject to the relevant requirements, Article 14 of the European Union’s AI Act requires more than nominal human involvement. It requires that high-risk AI systems be designed so they can be effectively overseen by natural persons during use, with oversight measures proportionate to the risks, autonomy level, and context of use. Those measures must enable assigned persons, as appropriate and proportionate, to understand system capabilities and limitations, monitor operation, remain alert to automation bias, interpret outputs, disregard or override them, and intervene in or interrupt operation. Article 26 separately requires deployers to assign oversight to persons with the necessary competence, training, authority, and support. Those details move the concept from human presence toward operational capacity. The point is broader than any one statute: human oversight is not meaningful merely because a human can be identified. The oversight role must be capable of being exercised. 

Contracts Present the Same Problem 

AI contracting increasingly includes provisions addressing training restrictions, confidentiality, human oversight, security, testing, monitoring, accuracy, agents, and prohibited uses. These protections can be important, but their effectiveness often depends on operational detail outside the sentence itself. Consider a promise that a vendor will not use customer data to train AI models. The practical effect may turn on:

• The definition of customer data, including prompts, outputs, feedback, telemetry, derived data, and embeddings;
• Whether the restriction covers product improvement as well as formal model training;
• Whether subprocessors receive the same restriction;
• Whether the restriction depends on a customer configuration or opt-out;
• What records and verification rights exist; and
• What happens if prohibited use occurs.

The contractual commitment is not rendered unimportant because those questions exist. The commitment supplies the legal obligation. The remaining provisions determine how closely that obligation fits the technical arrangement and what evidence will be available if a dispute later arises. The same analysis applies to provisions stating that an agent will operate only within authorized parameters. Useful drafting should identify the relevant parameters, who configures them, whether the agent can obtain additional permissions or invoke other tools, what actions require separate approval, how authority expires or is revoked, what the agent logs, and who bears responsibility for actions taken through customer-provided credentials or tools. The arrival of an accurate sentence is not the end of the work. It is often the first visible evidence that the work has been assigned, and a signal that the assignment now has to be operationalized. 

From Framework to Workflow 

Leading frameworks already point in this direction. The (officially) voluntary NIST AI Risk Management Framework organizes risk management around Govern, Map, Measure, and Manage, with governance operating across the AI lifecycle. Its Playbook and AI Resource Center provide suggested actions, references, and implementation resources. ISO/IEC 42001:2023 likewise defines an AI management system in terms of organizational processes used not only to establish AI policies and objectives, but to achieve, maintain, and continually improve them. The practical challenge is translation. Principles must become roles. Policies must become workflows. Contractual promises must align with architecture and configuration. Monitoring must reach someone with authority. Exceptions must alter future operation. 

Test the Control 

Frameworks can establish the architecture. Organizations still need to follow individual controls into the workflow and determine whether those controls operate. Begin with the sentence in the policy, contract, assessment, or governance framework. Then identify the operational elements we discussed above:

1. Trigger;
2. Actor;
3. Information available;
4. Actor’s authority and independence;
5. Timing;
6. Evidence produced;
7. Response to failure; and
8. Feedback mechanism.

Run a real or representative use case through the result. Ask where the control depends on someone noticing, remembering, interpreting, escalating, or disagreeing. Ask whether that person has the necessary information, time, authority, support, and incentive. Ask what the organization could later produce to show that the mechanism operated rather than merely that the mechanism was described. The gap between the sentence and the workflow is not proof that governance has failed. It is where the governance work begins.

For questions about AI governance program design, human oversight controls, or AI contracting, contact the Jones Walker Privacy, Data Strategy, and Artificial Intelligence team. Stay tuned for continued insights from the AI Law and Policy Navigator.