Governing AI on Databricks: the evidence ISO 42001 and the EU AI Act ask for

Every AI governance framework asks for broadly the same things: know what AI you run, control who can reach it, keep records of how it behaves, keep a human in the loop for the risky parts, and monitor it in production. The gap most organisations hit is not understanding the requirement. It is that the evidence lives somewhere other than where the AI actually runs, so producing it becomes a manual, point-in-time scramble.

If your AI runs on Databricks, a useful amount of that evidence can be generated by the platform as a by-product, rather than assembled by hand. This is not a compliance shortcut, and it is worth being precise about what the platform does and does not do. But the mapping is real, and it is worth understanding.

What the frameworks actually ask for

Strip ISO 42001 and the EU AI Act back to their operational core and you get a short list:

• An inventory. A maintained record of every AI system, model and tool in scope. ISO 42001 calls for an AI system register; the EU AI Act assumes you know what you operate and in what role.
• Access control. Who and what can reach each system, enforced, not asserted.
• Records and logs. An automatic trail of how a system was used, which the Act makes an explicit logging duty for high-risk systems.
• Human oversight. The ability to intervene, approve or stop, by design.
• Monitoring. Post-market monitoring of how the system behaves once it is live, not just how it tested.
• Data governance. Knowing where the data came from and that it was fit to use.

None of this is about a document. It is about being able to show the thing is true. That is where the platform earns its place.

Unity Catalog as the register and the access record

Unity Catalog is Databricks’ governance layer, and its relevance to AI governance is that it now registers and secures AI assets, models, agents, tools and MCP servers, alongside your data, under the same access controls, lineage and audit logging you already use for tables.

In practical terms, that gives you two of the items above in one place. It can serve as your AI system register, kept current as systems are added and changed rather than drifting out of date in a spreadsheet. And because every access operation is captured in audit logs, it is also your access-control evidence: not a claim that access was restricted, but a record of who reached what.

The Unity AI Gateway: controls, oversight and records at runtime

Registering assets governs them at rest. The harder half of AI governance is the runtime: what a model or agent actually does when it runs. Databricks’ Unity AI Gateway, built on Unity Catalog, is aimed squarely at that. It sits in front of models, agents and MCP servers and applies:

• Guardrails across prompts and responses, for example detecting PII, prompt injection, jailbreaks and unsafe content.
• Approval gates, so a human has to sign off before an agent takes a sensitive or irreversible action.
• Rate and cost controls, so usage stays within bounds.
• Payload logging, writing the full request and response of every call to a table.

Read that against the framework list and the overlap is obvious. Guardrails and approval gates are human oversight and control by design. Payload logging is the record-keeping duty, produced automatically. This is what governed agentic AI looks like when the platform does the enforcing rather than a policy document hoping people comply.

Evidence you can query, not assemble

The theme running through all of this is that governance evidence on Databricks is queryable. Unity Catalog audit logs, AI Gateway inference tables and MLflow traces all land in the lakehouse as tables. Lakehouse Monitoring watches data and model quality over time and writes its results the same way.

That changes the character of an audit. “Show me the records of how this system was used, who could access it, and how its behaviour was monitored” stops being a request that triggers a fortnight of manual collection, and becomes a set of queries against tables that were already being written. It is the same argument we make about evals as evidence: the strongest governance artefact is not a promise, it is a record you can produce on demand.

How it maps, briefly

The requirement	Where it lives on Databricks
AI system register / inventory	Unity Catalog asset registry
Access control and its evidence	Unity Catalog permissions + audit logs
Logging and record-keeping	AI Gateway inference tables, MLflow traces
Human oversight	AI Gateway guardrails and approval gates
Post-market monitoring	Lakehouse Monitoring
Data governance and lineage	Unity Catalog lineage

This maps to the ISO 42001 Annex A controls around AI systems, data, oversight and monitoring, and to the EU AI Act’s duties on record-keeping, human oversight, post-market monitoring and data governance. It does not map to all of either. The management system, the risk process and the human accountability sit above the platform, not inside it.

Where the platform stops

Two honest limits, because overstating this is how governance projects lose credibility.

First, the platform gives you the machinery, not the governance. Guardrails only help if someone decides which risks to guard against. Audit logs are only evidence if someone defines what good looks like and reviews against it. Tooling generates the artefacts; people and process still have to do the governing.

Second, none of this makes you compliant. Running AI on Databricks does not satisfy the EU AI Act, and it is not a substitute for an ISO 42001 management system or for EU AI Act readiness work. Certification of your management system does not, on its own, discharge the Act either. What the platform does is make the evidence cheaper to produce and harder to fake, which is a genuine advantage, not a finished answer.

The short version

If your AI runs on Databricks, treat the platform as an evidence engine for your governance programme, not as the programme itself. Unity Catalog gives you the register and the access record, the Unity AI Gateway gives you runtime controls and logs, and it all lands as queryable tables an auditor or customer can be shown. The work that remains is the work that was always yours: deciding the policy, setting the controls, and reviewing what the evidence tells you.

That is exactly the line we work on. We help teams put AI governance and an ISO 42001 management system around the tools they already run, so the evidence is designed in rather than reconstructed later. If you want to work out how your Databricks estate maps to ISO 42001 or the EU AI Act, talk to us.