AI Governance and AI Management Systems

Consider what an AI governance programme usually contains. Inventories, policies, risk assessments, model documentation, monitoring, evidence, certification — most of that is plan, do, check, act. The platforms are tooling for it: workflow, registers, evidence capture. The committee is usually a review body reporting into the same layer.

ISO/IEC 42001 is the interesting case, because it is not purely a management standard. Its leadership and planning clauses require an AI policy, AI objectives compatible with the strategic direction of the organization, allocated responsibilities and authorities, and risk criteria that separate acceptable risk from unacceptable. That is direction, and it sits inside the standard. Management review in clause 9.3 is oversight. Both halves are present.

What the standard does not do is address the governing body, and it says so itself, at exactly the two points where the decision is properly the board’s. The note to clause 5.2 refers organizations developing an AI policy to ISO/IEC 38507. The note to clause 6.1.1 refers the determination of how much risk the organization is willing to take or retain to ISO/IEC 38507 and ISO/IEC 23894. The line does not fall between the two standards. It runs through the middle of ISO/IEC 42001, and the notes are where the standard draws it.

The other half is smaller and more specific, and nothing in a compliance programme forces it. What will we use AI for, and what will we not? How much judgement are we prepared to delegate to a system nobody can fully explain? How much opacity is acceptable where an outcome affects a person? What will we spend, and against what return? Who answers when it goes wrong? Five questions, and no management system can answer any of them on the organization’s behalf.

That is the failure mode: management running ahead of direction. Management without direction optimises for the only goal anyone actually stated, which is usually evidence of compliance, so the inventory gets built, the policy gets written, the assessment gets filed, and none of it is aligned to an intent, because no intent was set. The documents pile up and nothing anyone does is different.

None of which makes the management work optional. It is most of the effort, most of what regulation requires, and most of what an auditor will examine. The point is narrower: it is the second half of the job, and doing it first is expensive.

ISO/IEC 42001’s own introduction makes two points this practice works to. An AI management system is an extension of existing management structures rather than a parallel one, which is why the answer to “do we need a separate AI governance function” is usually no. And an organization has to find an appropriate balance between governance mechanisms and innovation. Direction that only constrains is a failure of governance, not an excess of it.

Risk appetite is the decision that makes everything downstream possible, and ISO/IEC 38507 places it here deliberately. How much bias, uncertainty, opacity and autonomy the organization will accept is not a judgement the team building the system should be making on its behalf. Setting that boundary is a governing-body decision and not a delivery one: it is where an organization states how much it is willing to be wrong about, and in which direction.

The second standing obligation is visibility: a maintained, high-level view of every AI system in use, which function owns it, what risk it carries, and how it serves the strategy. That is the inventory again, arriving from the governance direction rather than the risk one, and it is the input without which the first verb cannot be performed at all.

Cadence and forum are part of the decision rather than administration around it. Who meets, how often, and with what standing inputs determines whether direction gets revised as the estate changes, or is set once at launch and quietly outlived. Reviewing project status and asking whether the direction still holds are different meetings. The second is the governing one, and it needs its own place on the agenda.

The standards are explicit. ISO/IEC 42001 requires roles, responsibilities and authorities to be defined and allocated (clause 5.3, and A.3.2 for AI specifically), and extends the requirement across partners, suppliers and third parties (A.10.2). The EU AI Act adds a sharper version for deployers: human oversight assigned to people with the competence and the authority to intervene. Authority is the word doing the work there: oversight becomes a control at the point the overseer can stop the system.

The definition the risk standards use is deliberately broad. A control can be a process, a policy, a device, a practice, a condition, or an action. It does not have to be a document, and with AI it usually is not: an access boundary, an input validation step, a logging configuration, a threshold that halts a model, and a mandatory human checkpoint are all controls, and none of them is written prose.

The standards also separate the control from the control objective. The objective is the outcome you are trying to secure; the control is the measure taken to secure it. ISO/IEC 42001’s Annex A is a table of both, which is why two organizations can share an objective and implement quite different controls against it, and why an auditor asks what you were trying to achieve before asking what you built.

Ethics arrives in both standards as a requirement rather than an aspiration. ISO/IEC 42001 clause 6.1.4 requires a process for assessing the potential consequences of AI systems for individuals, groups of individuals and societies — that one sits in clauses 4 to 10, and an audit tests it. Annex A then supplies the reference controls that implement it: the impact-assessment process and its documentation (A.5.2, A.5.3), and the assessment of impacts on individuals and on society across the lifecycle (A.5.4, A.5.5). Responsible development and responsible use have their own reference controls (A.6.1.2, A.6.1.3 and A.9.2, A.9.3). Whether each belongs in your control set is a Statement of Applicability decision — but excluding all of them while claiming a responsible-AI posture is a position that will not survive an audit.

NIST is no softer. GOVERN 1.2 requires the seven characteristics of trustworthy AI — valid and reliable; safe; secure and resilient; accountable and transparent; explainable and interpretable; privacy-enhanced; and fair, with harmful bias managed — to be integrated into organizational policies, processes, procedures and practices. Not published as principles. Integrated.

Nor is the content left to interpretation. ISO/IEC 42001's Annex C is informative rather than binding, but it sets out the objectives the standard has in mind, and points to ISO/IEC 23894 for how each one relates to risk management.

Specifying a control and enforcing one are different activities, and the gap between them is where most governance programmes are exposed. Written governance describes intent; production enforces behaviour. In between sits a policy requiring human oversight with no point in the workflow where a human can intervene, an approval requirement with no record that approval occurred, and a retention rule the pipeline never implemented.

The gap widens with autonomy. Agentic systems plan, call tools, and pass context between one another, so the decision path a control was written to constrain is assembled at runtime rather than at design time. Controls written for request-and-response do not bind a system that composes its own steps.

This is also where the data-side techniques earn their place — under the requirement they serve, rather than as a framework of their own. Provenance and lineage are what turn a claim about where training and retrieval data came from into a lookup instead of an assertion. Event logging and traceability are what turn a decision path into evidence instead of an intention. A defined intervention point, with authority attached and a record that it was available, is what stops human oversight being an entry on an org chart. Bounding the action space a system may operate within — which tools it may call, which data it may reach, which decisions it may take unaided — is what keeps an agentic workload inside something anyone can reason about afterwards.

Which of those you are legally obliged to produce depends on where you operate and what the system does. For high-risk systems within the EU AI Act’s scope the Act names them directly: data governance in Article 10, record-keeping in Article 12, human oversight in Article 14, accuracy, robustness and cybersecurity in Article 15. Outside that scope none of those articles binds, and the same capabilities are still what ISO/IEC 42001’s data and lifecycle controls ask for, and still what any usable answer to a customer, an insurer, a regulator in another jurisdiction or a court will require. The technique does not change. Only the reason you can be compelled to produce it does.

Designing that is architecture rather than paperwork, and it is the part that decides whether an audit is a formality or a problem.

A control that was designed correctly and is never tested is an assumption. Assurance is what turns it back into a control.

There are two questions, not one. Design effectiveness asks whether this is the right control for the risk — whether a human review step placed after the decision has already taken effect can prevent anything at all. Operating effectiveness asks whether it is working now, at the volume it currently runs at, with the people currently doing it. A control can pass one and fail the other, and most audit findings live in exactly that gap.

Cadence comes from the risk tier rather than the calendar. A system informing credit decisions and a system drafting internal summaries do not warrant the same test frequency, and treating them alike is how an assurance budget gets spent on the wrong things.

Evidence is the third piece, and it decides whether an audit is routine or a project. If demonstrating that oversight happened means reconstructing it from memory and email, the control may well have operated, and it cannot be shown to have.

Controls drift too, and more quietly than models do. Permissions widen. An exception path gets added for a deadline and never removed. A threshold is relaxed after a noisy quarter and never restored. None of that surfaces in a policy review, because the policy still says exactly what it always said.