An agent does not just answer. It calls tools, edits files, sends requests and spends money under an identity somebody granted it. That moves the governance question from “what did the model say” to “what did the agent do, and how do we know”. Here is a way to ask it that does not involve reading logs.
An agent’s own summary of what it did is a statement, not evidence. The harness transcript is better, but it is written under the same identity the agent runs as. Evidence is what the agent could not have rewritten: version control history, CI results, exit codes, egress logs, the bill. Keep the agent’s claim and the system record apart, compare them, and treat the difference as the finding.
For two years the AI governance conversation has been about outputs: is the answer accurate, is it biased, did it leak something, did a person review it before it went anywhere. Those questions still stand. But the systems now being rolled out do not stop at an answer. A coding agent opens a pull request. A workflow agent files the ticket, updates the record and sends the e-mail. An operations agent restarts the service. Each of these is an action, taken under credentials somebody issued, with side effects somebody will have to explain.
The scale is not hypothetical. In the EY AI Risk & Governance Survey of September 2026, 91 % of large firms (a US sample, pilots included) reported using agentic AI, 49 % had not updated their governance for it, 26 % said they could not detect unauthorised agents in their environment, and 36 % had already had a material AI incident. Read together, those four numbers describe an organisation where agents act faster than the controls were designed to observe.
In our own runs the agent’s summary is rarely false. It is incomplete: the diff is always longer than the story, which is why we read the diff first.
The people who notice this first are usually not in compliance. They are the engineers who granted the agent its token and then read a tidy summary of what it had done that did not quite match the repository. That gap is the subject of this page.
When someone asks “what did the agent do”, there are three places the answer can come from, and they are not equal.
| The record | Who produced it | What it is in a review |
|---|---|---|
| The agent’s own summary — the chat reply, the PR description, the “done” message | The agent | A statement. The actor chose what to say and what to leave out. |
| The harness transcript — every tool call with its parameters, every result | The runtime, under the same identity the agent runs as | Much better: it records what was attempted, not just what was reported. But the agent can usually reach the file. |
| Records outside the agent’s reach — version control history, CI results, exit codes, network egress logs, billing | Systems the agent cannot alter or bypass | Evidence. |
The rule that separates the third row from the first two is simple: a record is evidence only if the actor could not have rewritten it. That has nothing to do with whether the agent is honest. A truthful summary is still a summary. It becomes evidence only when something independent says the same thing.
Anyone who has worked in model risk will recognise the shape of this. Supervisory guidance on model risk management — SR 11-7 and its successor SR 26-2, which itself leaves generative and agentic AI outside its scope in the United States, and the local equivalents that borrowed from it — is built on independent verification and effective challenge: the developer’s account of a model is a starting point, not a conclusion. An agent describing its own run is the developer’s account. The system record is the validation.
Most environments keep only one of the three records, usually the first. That is why post-incident reviews of agent runs turn into archaeology: there is a narrative and there is a repository, and someone has to reconstruct what happened between them.
The cheaper pattern is to store the agent’s claim and the system record separately, labelled as what they are, and then compare them. Review stops being log-reading and becomes a diff. Where the two agree, nobody needs to look. Where they disagree, that disagreement is the finding, and it arrives with its own evidence attached.
A confident, well-structured review of the document it was asked to analyse — sections, findings, recommendations.
Record (systems outside the agent)Access log: the file was never fetched. The agent had no access to the text and wrote the review from the title and what such documents usually contain.
FindingThe summary was fluent; the record was empty. Nothing in the text gave it away. The access log did. Route to a person.
That one is from our own runs, with a hosted model, and it is the reason this page exists.
Nothing in that example required reading a transcript. It required that both records existed, were kept apart, and that somebody had decided in advance which one wins when they disagree.
These are questions, not thresholds. They are meant to be asked of the team that runs the agent, and a good answer to each one points at a record rather than at a person’s assurance.
Three of these questions are about the watcher, not the agent. Who can alter the configuration, who can alter the record, and who would notice a gap. Most agent-security material covers what the agent may do. Far less covers whether the account of what it did can be trusted, and that is the part an auditor will test.
Engineering teams push back on governance when it adds a step to their day. The controls that survive are the ones that make the record a by-product of running the agent, not a task performed afterwards.
We built a runtime proxy for agents with loop detection and a kill switch. The stop lived in the proxy, outside the agent’s process, which is the only place a stop can be trusted.
The EU AI Act compliance checklist walks one system through classification and then lists the obligations that attach to it, with the evidence an assessor usually asks for. Run the agent through it as a system in its own right, not as a feature of whatever it sits inside.
Open the checklistAn agent is an AI system. It belongs in the same inventory as every other one, with an owner who would recognise it if called, a stated purpose, a defined scope and a last-confirmed date. What is new is what attaches to the entry. For a model it was validation reports and monitoring results. For an agent it is the runtime records above — the wrapper’s start-and-end log, the diff against scope, the egress log, the triage decisions — and those are what someone will ask for when they ask what the agent did last quarter.
Whether a regulation obliges you to keep them depends on the system. For high-risk AI systems, Article 12 of the EU AI Act requires that the system technically allow automatic event logging over its lifetime, and that duty sits with the provider — which, for an agent you assembled from a general-purpose model and put to a high-risk use, may be you (Article 25). A harness transcript becomes that kind of log the moment it is written to a destination the agent cannot reach, which is also what makes it retainable. The deployer of a high-risk system has duties of its own under Article 26: to assign human oversight to people with the competence and authority to intervene — the oversight Article 14 requires the provider to design in — to monitor operation, and to keep the logs the system generates. Read against an agent, a kill switch and a runtime proxy are what “human oversight” looks like in practice, and the trajectory records above are the logs a deployer would be asked to produce. For everything else, which is most agents, there is no specific logging article to point to. The driver is your own audit, your customers’ due-diligence questionnaires and your own incidents — the 36 % in the EY survey did not have one because a regulation was missing. The question is the same in both cases: what did it do, and how do we know.
In Model Inventory for Jira, every AI system — models and agents alike — is a Jira work item with an owner, a purpose, a classification and a status, and reviews and incidents as linked work. The runtime records stay wherever your systems keep them; the inventory is where someone finds the owner, the scope and the last review when the question comes. The fields each entry needs are in our AI model inventory guide.
See how it worksAn LLM produces text, and the risk sits in what that text says; a person still decides what to do with it. An agent acts on the text under an identity someone granted it. The risk moves from the content of an answer to the effect of an action, and the control question moves with it: what did the agent actually do, what was it allowed to touch, and how would anyone know if it went further.
Yes, if they touch production code or production data, which is what makes them useful. A coding agent that commits, opens pull requests or runs deployments is a change actor in your delivery process, and every question you ask about a human change actor applies. The difference is that the agent writes its own account of the work, and that account is not evidence. The question is who can prove what the agent did without asking the agent.
Only for high-risk AI systems, under Article 12, and the duty attaches to the provider of that system. An agent is not high-risk by virtue of being an agent. Where it is high-risk, the deployer keeps the logs it generates and assigns human oversight under Article 26; the controls above are how that is done for an agent. For everything else there is no specific logging duty in the Act, but your auditor, your customer’s questionnaire and your own incident review will ask exactly the same question.
This page is a practical explanation, not legal advice. Confirm obligations against the official text of Regulation (EU) 2024/1689, the supervisory guidance that applies to you, and where the stakes warrant it, qualified counsel. Survey figures: EY AI Risk & Governance Survey, September 2026.