The Enterprise AI Agent Operating Model: Permissions, Owners, and Human Approval

An AI agent should not reach production until the company can answer a basic question: who is responsible when it takes action?

Not who bought the platform. Not who built the demo. Who owns the business result, the permissions, the review process, and the response when the agent does something wrong?

A lot of companies do not have that answer yet. They have a use case, a vendor, and an executive sponsor. Then the agent starts reading files, calling APIs, updating records, or sending messages across systems owned by different teams.

That is where the pilot becomes an operating-model problem.

The fix is not another AI policy nobody reads. Each production agent needs a small, specific control sheet that names its job, owner, identity, allowed actions, approval rules, evidence, and stop conditions. If those controls cannot be written down, the agent is not ready to act.

Start with the agent’s job, not the platform

An agent needs a narrow job description.

“Help finance” is not a job. “Collect approved department explanations, compare them with the monthly variance report, and prepare a draft summary for the controller” is much closer.

The job description should define:

What triggers the workflow
Which systems and data it may use
What output or action it may produce
Which cases it must escalate
What counts as a successful result
When it must stop

This sounds simple. It is not busywork. A vague job forces the agent to make choices the company never made.

The job should also identify the cost of being wrong. A bad internal draft is annoying. A bad payment, account deletion, production change, or customer promise can become an incident. Those workflows should not share the same approval model.

If you have not mapped the task yet, start with our guide on how enterprise AI changes workflows and decision rights. The operating model only works when the underlying job is clear.

Give every agent three named owners

Do not assign ownership to “the AI team.” That phrase usually means nobody knows who gets the call at 2:00 a.m.

Every production agent needs three forms of ownership:

Business owner: Accountable for the outcome, process rules, service level, and cost.
Technical owner: Accountable for identity, integrations, permissions, reliability, monitoring, and shutdown.
Risk owner: Accountable for approval boundaries, evidence, policy, incident handling, and periodic review.

One person can cover more than one role in a smaller company. The roles still need names.

The business owner cannot hand the entire problem to IT. IT can secure the connection and monitor the activity, but it cannot decide whether an agent should approve a credit, change a delivery date, or reject a claim. Those are business decisions.

The technical owner should not become the default reviewer for every output either. The person closest to the process should judge whether the work is accurate and useful.

NIST’s Generative AI Profile for the AI Risk Management Framework calls for clear roles and lines of communication for managing AI risk. It also recommends defining human oversight roles in the AI system inventory. That is a useful minimum standard: every agent should appear in an inventory with named people attached to it.

Create a separate identity for the agent

Shared user accounts are already a bad idea. Letting an agent borrow an employee’s broad access makes the problem worse.

A production agent should have its own identity whenever the platform and connected systems support it. That identity should make agent activity distinguishable from human activity. It should have only the rights needed for the assigned job, and its lifecycle should be managed when the agent changes or is retired.

Microsoft’s current documentation for Entra agent identities makes the reason clear. Organizations need to distinguish agent actions from workforce and workload identities, provide right-sized access, and prevent agents from reaching critical roles and systems. Microsoft also separates autonomous access granted directly to an agent from delegated access used on behalf of a person.

That distinction should be part of any platform evaluation, whether you use Microsoft enterprise services, Google Cloud, Salesforce, ServiceNow, or another agent platform.

Ask every vendor:

Does each agent receive a unique identity?
Can we see its owner or sponsor?
Can permissions be limited by system, data set, record type, and action?
Can access expire automatically?
Can we separate actions taken by the agent from actions taken by a person?
What happens to credentials and permissions when the agent is deleted?

If the answer is “the agent uses the employee’s access,” keep digging. That may be necessary for some delegated tasks, but it should not become invisible impersonation.

Build an action and approval matrix

Permissions answer what the agent technically can do. Approval rules answer what it is allowed to do in a specific situation.

Build a matrix for each workflow. Use four levels:

Read: The agent may retrieve approved information but cannot change it.

Prepare: The agent may create a draft, recommendation, proposed record, or change plan. A person releases it.

Act with approval: The agent may prepare an action and execute it only after the correct person approves.

Act within limits: The agent may execute a defined action automatically when the case stays inside documented boundaries.

Do not move directly from read access to autonomous action because the demo worked five times.

Approval should follow consequence. Customer communications, payments, contracts, privileged access, employee decisions, production changes, record deletion, and regulated data usually deserve stronger controls. Low-impact, reversible actions with tested error handling may earn more autonomy.

For every action, record the approver, evidence presented, approval timeout, escalation route, and rollback method. Decide whether approval must be per action, per batch, or per policy condition. “Human in the loop” is too vague to buy or govern against.

Log the evidence, not only the final action

A normal audit log may show that a record changed. An agent investigation needs more context.

You should be able to reconstruct:

Which agent and identity acted
Who triggered the workflow
Which tools, files, and systems it used
What action it proposed
What evidence supported the action
Whether approval was requested and who approved it
What changed
Whether the action succeeded, failed, or was rolled back

Logs also need retention, access, and redaction rules. Prompts, tool parameters, retrieved files, and model outputs may contain sensitive data. Sending everything into the SIEM without deciding who can see it can create a second problem.

Test the audit path before launch. Run a controlled failure and ask the security team to reconstruct it. If the answer depends on screenshots from the developer who built the agent, you do not have an investigation process.

Define stop conditions before the first incident

Every agent needs a way to stop safely.

NIST recommends protocols that allow generative AI systems to be deactivated when necessary. It also recommends go or no-go thresholds based on measured capabilities and risks.

Translate that into operating rules. Pause the agent when:

Error or correction rates cross the approved threshold
It requests access outside its job
A connector, model, instruction set, or data source changes without review
Logs stop reaching the monitoring system
The agent cannot identify the required approver
A security, privacy, legal, or customer-impact incident occurs

The technical owner needs authority and a tested method to disable credentials, stop scheduled runs, revoke connector access, and preserve evidence. A kill switch that nobody has tested is a comforting idea, not a control.

Also define how the agent returns to service. Require the owner to document the failure, fix the workflow, retest it, and approve the restart. Do not quietly turn it back on because the business is waiting.

Review the agent as a living service

Agents change even when the company thinks the project is finished. Models get updated. Connectors add actions. Permissions expand. Source data changes. Employees modify instructions. The business process itself moves.

Review each production agent on a schedule based on risk. The review should cover:

Current owner and business purpose
Identity and permission changes
Approval and exception history
Failed actions and human corrections
Cost per accepted outcome
Vendor, model, connector, and data-source changes
Incidents, near misses, and unresolved findings
Whether the agent should expand, stay limited, or be retired

NIST recommends ongoing review, after-action reviews, AI system inventories, and safe decommissioning. That last point matters. Retiring an agent should remove credentials, permissions, schedules, retained data, and dependencies without destroying records the company must keep.

Use this production gate

Before an agent moves beyond a controlled pilot, require a written yes to each question:

Does the agent have one narrow job and a measurable result?
Are the business, technical, and risk owners named?
Does it have a distinguishable identity and right-sized access?
Are allowed, blocked, and approval-required actions documented?
Can the team reconstruct an action from trigger through outcome?
Are stop, rollback, incident, and restart procedures tested?
Is there a scheduled review and retirement process?

A vendor’s admin console may support pieces of this. The vendor will not decide your accountability model for you.

Start with one agent. Make the ownership obvious. Keep the identity separate. Limit the permissions. Put consequential actions behind a real approval. Test the logs and the shutdown process.

Then let the agent earn more authority.

Catch Advisors helps IT leaders compare enterprise AI platforms and build vendor-neutral requirements around identity, governance, security, workflow fit, and measurable outcomes. If your team is evaluating AI agents, schedule a vendor-neutral assessment before a successful demo turns into an unmanaged production service.