A Checklist Before Letting an AI Agent Touch Real Work

There is a big difference between letting an agent organize your bookmarks and letting it near customers, money, or anything public with your name on it. Real work means real consequences, and agents need a safety setup that matches. This checklist covers the twelve things to verify before an agent touches anything that matters - each one explained with why it matters and how to actually check it, not just what to check. Run through it once per task setup - it takes about fifteen minutes and prevents the failures that take weeks to clean up. You will also find a risk-tier framework for calibrating how strict to be, and a worked hypothetical walkthrough showing the checklist applied to a realistic setup.

First: decide which risk tier this is

Not every agent task needs the full twelve checks at maximum strictness. Calibrate using three tiers - the tier sets how rigorously you apply everything below:

• Tier 1 - personal and reversible. Organizing your files, drafting your own documents, summarizing articles for yourself. The blast radius is you, and mistakes are undone in minutes. Apply the checklist lightly: skim it, confirm the obvious, move on.
• Tier 2 - shared but recoverable. Drafting team documents, preparing reports others will review, managing internal schedules. Mistakes waste other people's time and cause embarrassment, but nothing is irreversible. Apply the full checklist, with particular attention to instructions, limits, and the completion report.
• Tier 3 - external or irreversible. Anything touching customers, money, public communication, or production systems. Mistakes here cost reputation, revenue, or legal exposure. Apply every check strictly, complete three trial runs before launch, and keep a human approving every external action indefinitely.

The most common mistake is treating a tier-3 task with tier-1 casualness - usually because the task started as an experiment and quietly grew into production without anyone re-running the checklist. Whenever a task's scope, audience, or permissions change, re-determine the tier first; everything else follows from it.

Permissions and access

An agent can only damage what it can reach. The most effective safety measure is also the simplest: give it the smallest access that lets it do the job. An agent that drafts invoices does not need permission to send them. An agent that reads a shared drive does not need write access to it. Work through these four checks - each one closes a specific hole.

Check 1: the agent has read-only access wherever reading is all the task needs. Why it matters: most agent tasks are read-heavy - the agent reads ten things to produce one. Write access it does not need is pure risk with zero benefit. How to check: list every credential, key, and account the agent holds, and for each one ask what breaks if it becomes read-only. If nothing breaks, make it read-only. This single check eliminates entire categories of accidents, because an agent that cannot write cannot corrupt.

Check 2: it cannot send email, publish, pay, or delete anything without your explicit approval per action. Why it matters: external actions are the ones you cannot undo, and agents do not feel the weight of irreversibility - sending and drafting are equally easy operations to them. How to check: attempt the dangerous action in a test run, or inspect the permission scopes directly. The approval must be per action, not a one-time blanket grant - a blanket grant is just the dangerous permission with extra steps. For tier-3 tasks, make this structural (an account that cannot send) rather than merely instructional (a rule saying don't send), because instructions can fade mid-run but missing permissions cannot be exceeded.

Check 3: its access is scoped to specific folders, accounts, or projects - not your entire system. Why it matters: agents wander. Given broad access, an agent working on one folder will eventually open another - out of curiosity, by misreading a path, or because a tool returned more than expected. Scoping contains the wandering. How to check: verify that the agent's file access, API keys, and account permissions name specific resources, not wildcards. If you cannot enumerate exactly what the agent can reach, the scope is too broad. A useful test: ask yourself what the worst file the agent could touch is, and confirm it cannot reach it.

Check 4: you know exactly which credentials or API keys it uses, and you can revoke them in one place. Why it matters: credentials leak, get over-shared, and outlive their purpose. When something goes wrong at 11pm, you need a single switch that stops the agent immediately - not a scavenger hunt through three services. How to check: maintain a one-page inventory - which key, which service, what scope, created when, and the revocation link or command. If revoking would take you more than five minutes, consolidate first. Rotate keys when team members change or when a key has been pasted anywhere it should not have been.

What provider documentation states about permissions and retention

Two facts from official pages are worth building on, both checked 2026-11-03 and both specific to the products named. In the Claude Code permissions documentation, the default mode prompts for permission on first use of each tool, the plan mode lets Claude read files and run read-only shell commands without editing source files, and the bypassPermissions mode skips prompts except for a short list of actions no mode auto-approves. Rules are evaluated in the order deny, then ask, then allow, and the first match decides the outcome. A broad deny rule therefore cannot be undone by a narrower allow rule. The practical lesson for any agent product: write down what the agent must never do before you write down what it may do, and check whether your product evaluates denials first.

The same page shows a deny rule for a secrets file, written as Read(./.env), and also states that read and edit deny rules apply to the built-in file tools and to file commands Claude Code recognizes in shell commands, not to every command that might read a file. Treat a deny rule as one layer, and keep credentials out of the folder the agent works in as a second layer. Other agent products have their own rule systems, so read the permissions page of the product you actually use rather than assuming this behavior carries over.

Data exposure is the other half of access. Anthropic’s help page on how long it stores data, checked 2026-11-03, covers the consumer plans (Claude Free, Pro and Max), including Claude Code sessions run on those plans, and gives these periods: deleted chats leave back-end storage within 30 days, inputs and outputs flagged for usage policy violations can be kept up to 2 years (with trust and safety classification scores kept up to 7 years), and de-identified data can be kept up to 5 years when model training is enabled. The page says commercial products such as Claude for Work and the Anthropic API are covered in a separate article, not summarized here, so look up the API retention period there. The habit to copy is the lookup, not the numbers: before customer data goes to any tool, find that tool’s own retention and training-use page, write the period and the date you read it next to the credential inventory from check 4, and treat anything you cannot find as unknown rather than zero.

Instructions and limits

Vague instructions plus broad access is how agent incidents happen. Before real work, the instructions should be written down, reviewed, and tested - not improvised in a chat box five minutes before the run. These four checks ensure the instructions can actually carry the weight you are putting on them.

Check 5: the goal, constraints, and output format are written explicitly, not implied. Why it matters: agents follow what is written, not what you meant. Every implied expectation is a gap the agent will fill with a guess. How to check: read the instructions as a hostile literalist - someone determined to misunderstand. Anywhere you find yourself thinking well, obviously it means..., write it down. Pay special attention to the output format: specify it exactly, with an example, because format misunderstandings are the most common source of unusable output. The instructions should pass the stranger test: could someone who knows nothing about your work tell whether the output was correct just by reading the instruction?

Check 6: there is a stuck rule - what the agent does when something is unclear or a step fails. Why it matters: without one, the agent's default behavior on confusion is to guess quietly and keep going, which converts small problems into large ones. How to check: confirm the instructions name the exact behavior: try one reasonable alternative, then stop and report the step number, what was tried, the error received, and what is needed to continue. Then verify it works by deliberately testing with a broken input during trials - a stuck rule you have never seen fire is a hope, not a control.

Check 7: hard limits are set - maximum steps, maximum cost, and a time box for the task. Why it matters: limits are the circuit breakers that turn runaway loops from expensive disasters into cheap stopped runs. An agent with no step limit will retry a failing step indefinitely; one with a cost cap cannot surprise you with a bill. How to check: confirm all three numbers exist, are written in the instructions (not just in your head), and are sized to the task - a limit ten times larger than any legitimate run is no limit at all. Set them from your trial runs: take the longest successful trial and add a reasonable margin.

Check 8: the agent has completed at least three trial runs on low-stakes versions of the task without needing correction. Why it matters: one successful run proves the instruction is understandable; three prove it is reliable. Trial runs are also where you discover the agent's specific misreadings - the tool it misuses, the constraint it drops, the edge case nobody specified. How to check: keep a brief log of each trial - date, what went right, what went wrong, what changed in the instructions as a result. If a trial needed correction, the count resets; you need three consecutive clean runs. For tier-3 tasks, make at least one trial adversarial: feed it the messy input, the ambiguous case, the thing most likely to break it.

Oversight and rollback

Even good setups fail occasionally. The question is whether you notice quickly and can undo the damage. Oversight means you see what the agent did; rollback means you can reverse it. Both need to exist before the agent touches real work, not after the first incident. These four checks cover the after side of the equation.

Check 9: you receive a completion report for every run - what was done, skipped, and assumed. Why it matters: the report is your cheapest detection system. Skipped items reveal silent partial completion; stated assumptions reveal confident guesses before they become decisions. How to check: define the report format in the instructions and verify the first few reports actually contain all three sections - agents tend to omit the skipped and assumed sections unless explicitly required. Read the assumptions section with suspicion; it is where the agent confesses what it made up.

Check 10: you can review a log of the agent's actions, not just its final output. Why it matters: the output shows you the conclusion; the log shows you how it got there. Tool misuse, instruction drift, and wandering into out-of-scope areas all appear in the log long before they appear in the output - if they appear in the output at all. How to check: run a trial and confirm you can see each tool call with its parameters and results. If your agent product does not expose an action log, that is a significant gap for tier-2 and tier-3 work - consider it a missing check, not an inapplicable one.

Check 11: there is a way to undo the agent's changes - version history, backups, or draft states. Why it matters: detection without recovery is just watching damage happen in high definition. Every write the agent performs should be reversible within minutes. How to check: for each system the agent writes to, name the undo mechanism and test it once - restore a file from version history, revert a test record, discard a draft. Untested rollback is theoretical rollback. Pay special attention to systems where the agent's writes trigger downstream effects (notifications sent, workflows kicked off) - those need containment, not just file-level undo.

Check 12: someone is responsible for checking the agent's work on a schedule, even when it seems to be going well. Why it matters: agents degrade silently - instructions drift as tasks evolve, data formats change, edge cases accumulate. The setups that fail catastrophically are the ones nobody looked at for three months because they were working fine. How to check: name the person, the cadence (weekly for tier-3, monthly for tier-2), and what the review covers - a sample of recent runs, the failure log, and whether the instructions still match the task. Put it on a calendar. An unnamed, unscheduled responsibility belongs to no one.

A dry-run protocol you can repeat

Check 8 says how many clean trials to require. This protocol says how to run each one so that the result means something. It works for any agent product because it only needs the ability to run the task twice, once without side effects and once against a copy.

• Freeze a sample input set: ten real items with names and account numbers redacted, including at least two awkward ones (a missing field, an ambiguous request). Keep the same set for every trial so changes in results come from your changes.
• Write your own expected result by hand before the agent runs. A comparison against your memory of what you wanted is not a comparison.
• Pass one, intent only: remove or disable every write and send capability and ask the agent to list the actions it would take, in order, with the input it would use for each. Compare this list to your expected result.
• Classify every mismatch as an instruction problem, a permission problem or an input problem. Fix instructions for the first, permissions for the second, and add the input to the awkward set for the third.
• Pass two, against a copy: point the agent at a duplicate spreadsheet, a test mailbox or a scratch folder, let it write, and diff the copy against your expected result.
• Record the date, the input set version, pass or fail, and the change you made. Any change to instructions or permissions resets the clean-run count to zero.

The intent-only pass is the cheapest control in this guide, because it shows the plan before anything has happened. Its limit is that agents sometimes act differently from the plan they state, which is why pass two exists and why the action log in check 10 matters.

Cost caps in practice

Check 7 asks for a step limit, a cost cap and a time box. Set them from trial data, and add a cap that lives outside the instructions, because instructions can fade mid-run and an account-level limit cannot. For Claude Code, the Anthropic cost documentation, checked 2026-11-03, says Claude Console (API) organizations can set workspace spend limits, Team and Enterprise plans can set spend limits at the organization, group or member level once usage credits are on, and cloud-provider customers use their cloud’s own budget controls. It also says the dollar figure that /usage shows in Claude Code is computed locally from token counts and is an estimate, so use the billing console for authoritative numbers. For other providers, look for the equivalent page and write down the date you checked it.

A simple derivation: suppose three hypothetical trials took 22, 27 and 31 steps and 6, 8 and 9 minutes. Take the longest of each and add half again as margin. That gives a cap of 47 steps and 14 minutes per run (31 x 1.5 = 46.5, rounded up; 9 x 1.5 = 13.5, rounded up). A run that hits either cap should stop and report, not retry. Then set the account-level limit to a daily figure you could lose without a conversation about it, and review it when the task grows.

What to log, and who can read the log

Check 10 asks for an action log. A log that answers questions after an incident has these fields at minimum:

• Run identifier, start time, end time and the reason the run stopped: finished, step cap, time cap, stuck rule, or error.
• Which input items were processed, by identifier rather than by full content when the content is sensitive.
• Every tool call with its name, parameters and a success or failure status.
• Every approval requested, who answered, and how long the answer took.
• Every assumption the agent stated and every item it skipped, copied from the completion report.
• Token or cost estimate for the run, so a drift toward the cap shows up before the cap does.

A log can contain the very data you were protecting, so give it the same access limits as the source and set a deletion date. A log nobody may read is no better than none, and a log everybody may read is a second leak.

A second worked task: categorizing a bank export

This is a second hypothetical, an invented scenario. Imagine Marcus, a freelance bookkeeper, who wants an agent to suggest expense categories for the lines in a monthly bank export so that he can review them before the books are closed. Money is involved, so he treats it as tier 3 even though the agent never moves any.

Permissions and exposure: the agent never receives the bank login. Marcus downloads the export, removes account numbers and personal names from the description column, and gives the agent a copy of the file plus write access to one new spreadsheet. Before using any tool he looks up its retention page and notes the date. Reversibility: the agent writes to a new tab named suggested and never edits the original column, so undoing a run means deleting a tab, and the spreadsheet’s version history is the backstop. Review points: he reads every line the agent marks low confidence, plus a fixed sample of the rest, and nothing is posted to the ledger until he has done so.

Dry run: he takes three past months where he already knows the correct categories, runs the intent-only pass, then the copy pass, and sets the pass bar in advance: every line matches his categories, except lines the agent flagged. Refund lines, duplicate imports and a merchant renamed after a rebrand are the awkward items he adds to the frozen set. Caps and logs: step and time caps from the trials, a daily limit on the account, and a log with the fields above. The first real month he reads the log as well as the output. The point of the exercise is that each control is something he can name and test, which is what separates a checked setup from a hopeful one.

Failure modes and the signal that shows them

After launch, each way a checked setup goes wrong leaves a trace. Know the trace in advance:

• Scope creep: the log shows file paths or accounts outside the list in your inventory. Review the log for unfamiliar paths each week.
• A stuck rule that never fires: the completion reports show zero flagged items for many weeks in a row. Reinject a broken input on purpose once a month.
• Approval fatigue: nearly every approval is granted within seconds. Reduce how many approvals you ask for and make the remaining ones carry more detail.
• Cost creep: the per-run step count trends toward the cap. Investigate before raising the cap.
• Stale instructions: your corrections per week rise after a change in the data format or the task. Re-run the dry-run protocol.
• Log gaps: runs appear with no stop reason. Treat a missing reason as a failed run.

A worked hypothetical walkthrough

Here is a hypothetical example - an invented scenario, not a report of real events - of the checklist applied to a realistic setup. Imagine Priya, a hypothetical operations manager at a small online retailer, who wants an agent to draft replies to common customer support emails: order status questions, return requests, and shipping inquiries. The agent would read incoming support emails, draft replies, and save them for Priya's team to review and send. She runs the checklist before launch.

Tier first: customer communication is tier 3 - external, reputation-bearing, and the drafts go to real customers once approved. That means every check applies strictly, three clean trial runs are required, and human approval stays on every send permanently. Permissions: the agent gets read access to the support inbox and write access to a drafts folder - nothing else. It cannot send email (check 2, structural: the mail account it uses has no send permission), cannot access order or payment systems (check 3), and Priya lists its two credentials with revocation links in a shared doc (check 4).

Instructions: she writes the goal explicitly (draft a reply to each email, saved as a draft, categorized by type), leads with constraints (never send, never promise refunds beyond the published policy, never invent order details - mark unknowns), and includes a stuck rule (if the order number is missing or the request is unusual, draft nothing and flag for human review). Limits: 50 emails per run, stop after two consecutive failures on the same step. Trials: three clean runs on last week's emails, including one adversarial trial with an angry customer, a missing order number, and a request outside the return policy - the cases most likely to produce a bad draft.

Oversight: every run ends with a completion report (drafted, skipped, flagged - with reasons), Priya reviews the action log for the first two weeks, the drafts folder has version history so any bad draft can be discarded, and she puts a weekly 20-minute review on her calendar to sample drafts and update instructions. The walkthrough surfaces two gaps she would have missed otherwise: the refund-policy boundary (an instruction gap, fixed before launch) and the missing order number case (a stuck-rule gap, found in the adversarial trial). That is the checklist earning its fifteen minutes.

After the first real run

The checklist doesn't end at launch. Treat the first few real runs as an extended trial: review everything, keep notes on every surprise, and tighten the setup after each one. Most teams discover that their instructions need two or three rounds of fixes before the agent is genuinely reliable - usually around edge cases nobody thought to specify. That is normal. What matters is that the surprises happen while you are watching closely, with rollback ready, rather than months later when nobody is checking.

• Review the first five real runs completely, not just the summaries.
• After each surprise, add one instruction or limit that would have prevented it.
• Widen the agent's scope gradually - one new responsibility at a time.
• Re-run this checklist whenever the task, tools, or permissions change.

Build the habit of treating every surprise as a checklist item in waiting. The surprise that gets logged and patched makes the setup stronger; the surprise that gets shrugged off becomes the incident report later. Over time, the checklist runs themselves get faster - the second time through takes ten minutes, the third takes five - because the inventory, the limits, and the rollback procedures already exist. The upfront cost is real but one-time; the protection compounds.