When someone asks why an AI-assisted workflow changed a customer record, a long chat transcript may still fail to answer. It can show what the assistant said without proving which tool ran, who approved the exact payload or whether the target system saved the change. Keeping more conversation is not automatically better evidence.
A proportionate audit design begins with the questions an authorised reviewer needs to answer. This article proposes an event schema for a record-update workflow. It focuses on action evidence, access and retention decisions, rather than setting a legal retention period or claiming that references and hashes make customer information anonymous.
Define the questions the audit must answer
For a disputed update, the reviewer may need to identify the initiating actor, target record, proposed field, approval, tool operation and verified outcome. They may also need the applicable policy version and evidence supporting the proposed value. That list guides the event design more precisely than a rule to log everything.
Separate operational investigation from unrestricted content access. A support colleague can often identify a failed tool operation using its reference and status without reading the underlying customer message. A qualified investigator with appropriate access may need the source later. Those are different permissions and should remain different paths.
Decide what evidence is actually necessary for each action type. A denied search might require the reason category and actor, but not the denied record's full details. A successfully verified update may require a restricted value comparison. Record the purpose of retaining that comparison and where an authorised reviewer can obtain it.
Log the action stages as trusted events
OpenAI's function-calling guide distinguishes a model's request from application execution and the returned output. Use that distinction to label events: a requested update is not an executed update, and an executed operation is not necessarily a verified business result. Source: OpenAI function calling
The trusted application should supply the authenticated actor, approval reference, timestamp and execution outcome. Do not let the model assert that a manager approved something and then save that assertion as the authoritative approval record. Model-generated interpretation can be useful, but label it as interpretation with its supporting reference.
A request that is rejected deserves an explicit denied event. A timeout after a write needs an uncertain outcome until reconciliation determines whether the write occurred. Do not collapse these states into a single failed label and retry blindly. The event stream should make the next investigation step visible.
Proportionate audit-log schema
Use this proposed schema for a customer-address update. Fields listed as restricted evidence stay outside the general event stream unless the information owner specifically approves that content there.
| Field | Value or rule | Trusted source and purpose |
|---|---|---|
| event_reference | Unique application-generated reference | Identifies this event without exposing conversation content |
| occurred_at | Timestamp with timezone or UTC offset | Application clock; supports sequence reconstruction |
| workflow_reference and version | Stable workflow instance and deployed rule version | Application; identifies the policy and execution being investigated |
| actor_reference | Authenticated initiating identity, using the approved internal identifier | Authentication service; never a name inferred by the model |
| action_stage | requested, denied, approved, execution_started, execution_reported, verified or unresolved | Application state transition; distinguishes intent from outcome |
| target_reference and permitted_field | Restricted record reference and allowed field name | Access-checked request; enough to locate the action without dumping the record |
| proposal_reference | Exact payload reference or controlled integrity value | Application proposal store; binds approval to the intended change |
| evidence_reference | Restricted source reference and relevant version | Evidence store; source content remains subject to separate access checks |
| decision_reference and reviewer_reference | Exact approval or denial record plus authenticated reviewer | Trusted review system; absent when no reviewer decision exists |
| tool_operation_reference | Connected service operation reference, when available | Tool adapter; supports readback and reconciliation |
| outcome and reason_category | Verified success, denied scope, validation failure, conflict, unavailable verification or other approved category | Application verification; avoid raw error dumps containing personal data |
| retention_class and access_class | Owner-approved handling categories for this event and linked evidence | Information-owner policy; no universal duration implied |
| interpretation_reference | Optional separate model summary marked as non-authoritative | Restricted interpretation store; not a substitute for the events above |
For a fictional TEST-ACCOUNT-A update, record separate events for request received, exact proposal approved, execution started, tool result received and permitted field verified. Keep the synthetic original and replacement addresses in the restricted fixture store, with their proposal reference in the event stream. A denied TEST-ACCOUNT-B request records denied scope and no execution reference. An uncertain TEST-ACCOUNT-C timeout records unresolved status and a reconciliation owner.
Include these review rules in the implementation brief:
- Required event fields are validated before saving. Missing actor or decision evidence cannot be repaired by asking the model to guess.
- Approval binds the target, field, payload and applicable version. A changed payload requires a new decision and event.
- An operation reported as successful is checked against the authoritative system before the verified event is emitted.
- Sensitive raw errors, prompts, attachments and credentials are excluded from the general event stream. Necessary evidence uses restricted references.
- Retention and access for event rows, proposal records and source evidence are decided separately and tested together.
Acceptance is demonstrated with normal execution, missing approval, duplicate request and uncertain-result fixtures. Reviewers must be able to reconstruct each actual outcome and inspect necessary evidence only with authorised access. Check that deleting or restricting linked evidence behaves according to the approved handling decision rather than leaving unexpected copies in exports.
Keep evidence references useful and controlled
A reference is only useful if an authorised reviewer can resolve it to the right version. A broken link or overwritten source may prevent reconstruction. Decide which versioned evidence must remain available, for which investigation purpose and under whose approved handling schedule.
Do not assume that hashing a payload removes its privacy implications. A hash can still link events to a person or reveal matches when candidate values are predictable. Treat integrity values and stable record references according to their actual use and exposure. Avoid putting personal details in the reference label, URL or filename.
Also test what a lower-access reviewer sees. They may receive an event's permitted status and an instruction to request authorised investigation, rather than a convenient link exposing the source. Exported logs and dashboard previews need the same access review as the primary event store.
Reconcile local retention with provider handling
OpenAI's data-controls documentation describes abuse-monitoring logs and application state, including endpoint-specific conditions and feature limitations. Your event-log design controls your own stored records; it does not establish that the provider or a connected service retains no content. Source: OpenAI data controls
Map the source document, model request, tool adapter, review system, general events, restricted evidence, backups and exports. Ask the information owner which records are necessary and who sets their handling decisions. Verify actual deletion and access behaviour for the selected systems; do not promise a blanket period simply because one component has a configurable setting.
Structured output can help keep event summaries consistent. OpenAI documents schema-constrained responses and refusals, but the schema does not verify that the event happened or that a person truly approved it. Populate authoritative facts from trusted systems and keep model-generated fields separately labelled. Source: OpenAI structured outputs
Investigate normal, missing and duplicate outcomes
In a hypothetical normal case, the approved proposal executes and readback confirms the intended field value. The reviewer follows the proposal, approval and operation references to reconstruct the change. They do not need the full conversation in the general log to determine the execution sequence.
In a missing-evidence case, an approval reference cannot be resolved. The event should record that gap rather than presenting the update as properly approved. The owner investigates the trusted review system. If the operation has not executed, it remains held; if it has, the gap needs investigation rather than a retrospective invented approval.
In a duplicate case, the same proposal is requested twice. The application can return the existing operation result under the proposed duplicate-handling rule, while the log records both requests and one execution. If a timeout leaves the first outcome uncertain, reconciliation comes before another execution. This makes uncertainty visible without storing every repeated customer utterance.
FAQ about useful and proportionate AI audit logs
Should we save a full transcript for every action?
Decide from the investigation purpose and approved handling requirements. The proposed schema supports action reconstruction with references and trusted events. Where source content is necessary, retain it through the appropriate restricted evidence process rather than duplicating it indiscriminately.
Can an AI-generated explanation be the approval evidence?
No. It can interpret a decision record, but approval evidence should come from the authenticated review system and bind to the exact action. Preserve that difference in the schema and reviewer interface.
What should the log say when an update might have succeeded?
Record the attempted operation and unresolved verification, with a reconciliation owner. Avoid claiming either confirmed success or confirmed non-execution until the authoritative system resolves the outcome.
If your business needs help defining this process, explore Custom AI agents, the wider AI automation services, and our custom-agent workflow guide. The agents and automation comparison and custom AI agent glossary explain the terms. To discuss your records and approval rules, get in touch.