Explainability Is for You. Evidence Is for the Regulator.

Why one cannot substitute for the other

An explanation can be regenerated on demand. Run the tooling again and you get an explanation again, possibly a slightly different one, because the explanation is derived from the model rather than recorded from the event.

Evidence has to have existed at the moment of the decision. It has to be bound to the version of the policy that was in force then, not the current one. It has to be attributable to a named owner. And there must be no path by which it could have been skipped.

That last property is the one most systems lack, and it is the one that actually gets tested. A control that could have been bypassed without leaving a trace is not a control. It is a convention.

What is Evidence by Design?

Evidence by Design is the architectural principle that consequential AI outputs should carry the evidence, provenance, confidence and caveats needed to defend them, generated as the system runs rather than assembled afterwards.

The reconstruction trap

Here is the failure that actually happens.

The supervisory question arrives. It is narrow and reasonable: why did this system reach that outcome, on that date, for that customer?

Somebody begins assembling the answer. They pull logs from three systems, reconstruct which model version was serving, find the policy document as it stood, and interview the two people who remember the change.

Whatever they produce, it is not evidence. It is an argument, built after the question, by someone with an interest in the outcome. Everyone in the room understands the difference, including the examiner.

You cannot reconstruct evidence afterwards. That is the entire point of it.

What has to be captured

The minimum set is smaller than people fear.

The policy version in force. Bind it to the decision at the moment it is made. This single field converts a log into a record, because it makes "which rule applied" answerable by query instead of by memory.

A named owner. Not a team, not a service account. Accountability that resolves to a person is what a supervisor is actually looking for.

The inputs that mattered. Not the entire context window. The inputs the decision turned on, which is a design decision rather than a logging volume decision.

The path not taken. Whether a control fired, and whether escalation was available and declined. Absence of evidence that a control ran is indistinguishable from the control not running.

The cheap version

There is a version of this that costs almost nothing, and most organisations already have most of it.

The cost ledger is the underrated one. It already records who ran what, when, and how much it consumed. Add the policy version in force and join each tool call to a named control, and a finance artifact becomes an audit artifact. It was built for a different reason, which is exactly why nobody thought to skip it.

The expensive version is the one where capture is retrofitted during an examination that is already going badly.

The reframe

Explainability answers a question you are asking yourself while you build.

Evidence answers a question somebody else asks you later, under conditions you do not control, about a decision you may not remember.

Design for the second and you get the first as a by-product. Design only for the first and you discover, at the worst moment, that you have a great deal of insight into your model and no record of your institution.

© 2026 Nabeel Khan. Explainability Is for You. Evidence Is for the Regulator. is published under CC BY-NC-ND 4.0. Quote it, cite it, do not repackage it.