AI Governance That Survives an Audit: Controls, Trails, Ownership

AI governance is useful when it changes how a system is designed and operated. A policy that nobody can connect to a deployment, a review decision, or an incident is difficult to test and harder to defend. The practical question is not whether a document exists, but whether ownership and evidence exist at each control point.

Define roles and risk classes

Assign an owner for the workflow, a technical owner, a data owner, and a person accountable for consequential review. Separate proposal, approval, deployment, and incident roles where the risk justifies it. A small system may combine roles, but the combination should be explicit.

Risk classification should follow consequence and reversibility. A drafting assistant, a retrieval system exposing internal records, and an agent able to change a business system need different controls. Classification is a routing decision: it determines review depth, logging, access, evaluation, and escalation.

Govern data and models at the boundary

Document purpose, data categories, source authority, retention, redaction, and provider egress. Retrieval must enforce permissions at query time; hiding a forbidden source after generation is too late. Keep prompts, schemas, model versions, provider settings, and tool policies versioned so a change can be reviewed as an engineering artefact.

Provider and vendor risk includes availability, model changes, data handling, region, rate limits, and exit options. A fallback is only real when its quality, cost, and data behaviour have been evaluated. Do not treat a provider label as a governance control.

Controls, logs, and evaluation

Controls should be testable: allowlists for tools, schema validation, approval gates, source requirements, budget policies, and redaction. Define what happens when a control fails. A safe default may be a retry, a queue for review, a refusal, or a rollback; it should not be an unlogged continuation.

An audit trail records enough context to reconstruct a decision without copying unnecessary sensitive content: request identifier, actor or service identity, source or retrieval reference, prompt and model version, policy result, output status, reviewer action, timestamp, and downstream effect. Hashes and references can be preferable to raw payloads. Retention and access to the log are governance decisions too.

Evaluation connects governance to quality. A golden set can test factual grounding, permissions, unsafe tool requests, refusal behaviour, and output structure. Release gates define thresholds and exception approval. Production monitoring adds drift, abuse, latency, cost, and user feedback signals.

Human oversight and incidents

Human oversight is more than a button. The reviewer needs enough context, a clear decision vocabulary, a reason field, and a route for escalation. High-impact actions should not be batch-approved by default. Incident handling should cover unsafe output, data-boundary failure, provider outage, prompt injection, and unexpected spend. The playbook states how to contain, preserve evidence, notify owners, remediate, and verify recovery.

A workable operating loop

Start with one workflow and map its controls. Run abuse cases and ordinary evaluations. Review the resulting evidence with technical, process, and responsible stakeholders. Ship only the controls that have an owner and a failure path. Revisit the classification when the model, data, tools, users, or consequence changes.

This is an operating pattern, not legal advice or a claim of compliance with a specific regime. The Security service, audit-trail insight, and data-access use case cover related implementation details.

Make review evidence proportional

Not every low-risk drafting call needs the same trail as a tool action affecting a customer record. Proportionality can mean shorter retention, fewer fields, or sampling, but it should never mean losing the ability to explain the control decision. Define the minimum evidence for each risk class and test whether an independent reviewer can follow it without relying on the original operator’s memory.

Governance also needs a change calendar. New providers, tools, data sources, model versions, and prompt contracts can change the risk class. A lightweight review at each such boundary keeps the operating model current without turning every edit into a committee process.