74% of companies plan to deploy AI agents in 2026. Only 21% have any governance framework for them. The other 53% are running autonomous systems that can send emails, delete files, call APIs, and make decisions, with no written policy on what they are authorized to do.
This template fixes that. Copy the sections below into your team wiki. Customize the bracketed fields. Have the engineering lead and a senior stakeholder sign off on it. That is all it takes to have a documented AI agent governance policy.
TL;DR: A workable AI agent governance policy for small teams needs five things: an agent authorization register (what agents exist and what they can do), a data scope agreement (what data each agent may access), five hard red lines (actions that always require human approval), an audit log format (what to record when an agent acts), and an incident escalation path. This template covers all five in copy-paste form.
Why AI agents need a separate policy from your AI acceptable use policy
Your AI acceptable use policy covers what employees do with AI tools. ChatGPT, Claude, Copilot: a human types a prompt, reviews the output, decides what to do with it.
AI agents are different. An agent takes action without a human reviewing each step. It reads your inbox and drafts replies. It runs a code deployment. It queries your database and sends a report to a customer. It books a meeting on your behalf.
The accountability gap is real. If an agent sends a wrong external email, that happened. If it deletes a row from your database, that happened. You cannot unsend or undelete. And if you have no documented policy saying what that agent was authorized to do, you have no defense when a customer, auditor, or regulator asks who approved it.
Section 1: Agent authorization register
Every AI agent your team runs must be registered. This is the master list.
Template: copy into your team wiki
AGENT AUTHORIZATION REGISTER
Last reviewed: [DATE]
Reviewed by: [NAME, ROLE]
| Agent ID | Agent Name | Purpose | Owner | Deployment Date | Status |
|---|---|---|---|---|---|
| AGT-001 | [e.g., customer-support-bot] | [e.g., draft responses to tier-1 support tickets] | [Name] | [Date] | Active |
| AGT-002 | [e.g., pipeline-deploy-agent] | [e.g., run staging deployments on PR merge] | [Name] | [Date] | Active |
| AGT-003 | ... | ... | ... | ... | ... |
Instructions:
- Every agent, including internal automation scripts that call LLMs, must appear on this register
- "Agent" means any system that takes action autonomously using an LLM or AI model
- The owner is accountable for that agent's behavior
- Review the register quarterly; remove decommissioned agents within 30 days
Section 2: Data scope per agent
Each registered agent gets a data scope agreement: an explicit statement of what data it may access and what it may not.
Template:
DATA SCOPE: [Agent ID]: [Agent Name]
PERMITTED DATA ACCESS
- [e.g., Support ticket content (text only, no attachments)]
- [e.g., Customer name and ticket ID]
- [e.g., Internal knowledge base articles]
PROHIBITED DATA ACCESS
- Customer payment data, credit card numbers, bank details
- Employee personal records, salary, performance reviews
- Health information
- Authentication credentials (passwords, private keys)
- Data belonging to other customers (no cross-tenant access)
DATA RESIDENCY
- Agent may only process data in: [e.g., EU/US/your jurisdiction]
- Agent may not transmit data to: [e.g., third-party APIs outside the approved vendor list]
RETENTION
- Agent-processed data is not stored beyond the session unless explicitly logged
- Audit logs are retained for [e.g., 90 days]
Fill in one scope block per agent. If an agent needs access it does not currently have, that requires a change request reviewed by [NAME/ROLE].
Section 3: The five red lines
These five actions are prohibited for all AI agents without explicit human approval, regardless of the agent's confidence or the context.
Red line 1: External communication An agent may not send any message to an external party (email, Slack to an external workspace, webhook to a third-party service, API call that triggers a notification) without a human explicitly approving that specific message before it is sent.
Why: Automated external messages create legal and reputational liability. A mistaken email to a customer cannot be recalled.
Red line 2: Irreversible data deletion An agent may not delete, overwrite, or truncate data if there is no rollback path available. Soft deletes (mark as deleted, retain for 30 days) with a human-accessible restore path are permitted. Hard deletes are not.
Why: Data loss is one of the most common AI agent incidents. Agents optimizing for task completion will delete what is in the way.
Red line 3: Financial commitment An agent may not make any purchase, commit to any subscription, initiate any bank transfer, or approve any invoice. Financial actions require a human with spending authority.
Why: Agents can trigger real payments. An agent managing vendor APIs could inadvertently subscribe to a paid tier or trigger metered billing.
Red line 4: Account and permission changes An agent may not create user accounts, modify user permissions, add OAuth grants, or change role assignments. These actions require a human with admin access.
Why: Permission changes are the highest-impact reversibility failure. A misconfigured permission can expose your entire system.
Red line 5: Out-of-scope data access An agent may not access data outside its defined data scope (Section 2), regardless of whether it can technically reach that data.
Why: Agents that can read your filesystem can read things you never intended. Data scope is an explicit constraint, not a best effort.
How to enforce red lines:
- Build technical guardrails into the agent's tool configuration; do not rely on prompt instructions alone
- For every tool the agent has access to, verify: can this tool trigger a red line action? If yes, add a confirmation step before the agent calls it
- Log every action that touches a red line boundary, even when properly blocked
Section 4: Audit log format
Every agent action must be logged. Here is the minimum required format.
Log entry fields:
{
"timestamp": "[ISO 8601, UTC]",
"session_id": "[unique per agent session]",
"agent_id": "[from the authorization register]",
"agent_version": "[semantic version or commit hash]",
"action_type": "[one of: read, write, delete, external_call, escalated_to_human]",
"action_target": "[what was acted on: file path, API endpoint, table name, etc.]",
"input_summary": "[brief, non-sensitive description of what triggered this action]",
"output_summary": "[brief description of what was produced or changed]",
"human_approved": [true/false],
"human_approver": "[name or null if autonomous]",
"outcome": "[one of: success, failed, blocked, escalated]",
"error": "[null or error message]"
}
Storage requirements:
- Logs must be written to a location the agent itself cannot modify
- Retain logs for at least 90 days
- Make logs searchable by session ID and agent ID
- Your security lead must be able to access logs without involving the agent
What to do with the logs:
- Review weekly: any
outcome: blockedentries indicate the agent tried something it should not have. Investigate each one. - Review monthly: any pattern of
action_type: external_callwithhuman_approved: falseshould be checked against red line 1 - After any incident: reconstruct the session using
session_idto understand the full action sequence
Section 5: Human-in-the-loop triggers
Beyond the five red lines, define when agents should pause and ask a human, even for actions that are technically permitted.
Template: escalation thresholds
HUMAN-IN-THE-LOOP TRIGGERS FOR [Agent Name]
The agent must pause and request human approval before proceeding if:
- [ ] The task affects more than [N] records at once
- [ ] The confidence score for the action is below [X]%
- [ ] The action would modify data in [specific tables/systems]
- [ ] The agent has attempted the same action more than [N] times without success
- [ ] The task involves a customer account flagged as [high-value / enterprise / at-risk]
- [ ] [Add triggers specific to this agent]
Set these thresholds conservatively at first. It is easier to relax a threshold than to explain to a customer why an agent made a decision you did not intend.
Section 6: Incident escalation
When an AI agent causes an unintended outcome, here is what to do.
Step 1 (immediate): Contain
- Stop the agent session if it is still running
- If the agent sent external communications, note what was sent and to whom; do not send corrections yet
- If data was deleted or modified, check whether a rollback is available
Step 2 (within 1 hour): Assess
- Pull the audit log for the session using the
session_id - Identify which action caused the problem and what triggered it
- Determine whether a red line was crossed without the required human approval
Step 3 (within 4 hours): Report
- Notify [SECURITY LEAD] and [DATA PROTECTION OFFICER if applicable]
- If personal data was involved and you operate under GDPR, start the 72-hour clock for potential breach notification assessment
- Document the incident in your incident log with: what happened, what data was affected, what customers were affected, and what the agent was authorized to do
Step 4 (within 1 week): Fix
- Update the agent's tool configuration or data scope to prevent recurrence
- Update the audit log threshold triggers if the incident was a near-miss
- Run a blameless post-mortem with the agent owner
Implementation checklist
Copy this into your project management tool:
- Complete the agent authorization register (Section 1) for all currently running agents
- Write data scope agreements (Section 2) for each registered agent
- Verify the five red lines are technically enforced in each agent's tool configuration, not just in the system prompt
- Configure audit logging (Section 4) for every agent; confirm logs write to a location the agent cannot modify
- Set human-in-the-loop escalation thresholds (Section 5) for each agent
- Define the incident escalation path (Section 6) and make sure the security lead knows it
- Schedule a quarterly register review
- Brief the engineering team on the red lines and the audit log format
Quarterly review process
AI agents change fast. Set a calendar reminder for every three months to:
- Review the authorization register: are there agents that are no longer used? Remove them. Are there new agents that were not registered? Add them.
- Review data scope agreements: has any agent's access crept beyond its documented scope? Fix it.
- Review blocked log entries: patterns here indicate the agent is being asked to do things outside its authorization. Either update the agent's task definition or update the authorization.
- Review red line triggers: has anything changed that makes the current thresholds too loose or too strict?
The policy itself should be version-controlled. Update the version number and date at the top of the document whenever a material change is made, and notify the engineering lead.
What this policy does not cover
This policy governs AI agents your team operates. It does not cover:
- AI tools employees use directly (covered by your AI acceptable use policy)
- AI systems embedded in third-party SaaS products your team subscribes to (covered by your vendor due diligence process)
- AI models you train or fine-tune (requires a separate data governance and model card process)
For a full governance framework that covers all three, see the AI governance guide for small teams linked below.
