What You Are Accountable For When a Model Decides

How automation changes financial controls

Finance functions are adopting machine learning and large language models for work that used to consume qualified time: coding transactions, matching invoices to purchase orders, drafting reconciliation explanations, flagging anomalies, summarising contracts for revenue recognition. The productivity gain is real and, in routine high-volume processing, substantial.

What is less examined is that a control has changed character. Where a person used to make a determination that another person reviewed, a model now makes it. The output looks the same. The accountability has not moved at all, but the evidence supporting it has.

The review that disappears with manual work

A mill installs an automatic sifter to replace hand-sorting of grain. Throughput rises and consistency improves — a genuine advance. But the miller who used to notice, by feel, that a consignment had come in damp now sees only clean output.

The sifter did not remove the need to know whether the grain was damp. It removed the mechanism by which the miller happened to find out. Someone must now check moisture deliberately, because nothing checks it incidentally any more.

Control objectives, systematic risk and accountability

A control is an activity that someone performs, that leaves evidence, and that would be noticed if it stopped. Automating the activity satisfies the first element more consistently than a human ever could. It does not automatically satisfy the second or third — and it silently removes the incidental review that human processing provided for free.

Three consequences follow. First, the control objective is unchanged. If the objective was that expenditure is recorded in the correct period and account, that objective still needs to be met and evidenced; "the model classified it" is a description of the process, not evidence of the outcome.

Second, the risk profile changes shape. Human error is distributed and self-limiting — one clerk’s mistake affects one clerk’s transactions. Model error is systematic: a mis-specified rule or a shifted input distribution produces the same wrong answer across the whole population, at once, consistently, and without anyone hesitating. Consistency, which is the principal benefit, is also the principal risk.

Third, accountability is not delegable to a system. Responsibility for the financial statements rests with management under the Companies Act; responsibility for the audit opinion rests with the auditor under the Standards on Auditing. Neither framework contemplates a model as a responsible party, and neither is modified by the fact that a tool performed the work well.

Design review, traceability and monitoring

For every automated determination, identify what it replaced and what now performs the equivalent review. That is usually a designed control — sample testing of model output, exception thresholds, or reconciliation to an independent total — and it must be documented as a control, with a named owner and retained evidence.

Insist on traceability before deployment, not after an incident. For any output that feeds the financial statements you should be able to reconstruct which inputs produced it, which version of the model was in use, and who approved that version. A tool that cannot support this is usable for drafting and analysis; it is not usable inside the financial reporting chain.

Monitor for drift explicitly. Set an interval, re-test against known-correct outcomes, and record the result. A model that was accurate at implementation is not evidence of a model that is accurate now — the business changes, the inputs change, and nothing about the output announces that it has degraded.

And be clear about which questions you will not automate. Going concern, management estimates, related party characterisation and the tax positions discussed elsewhere on this site turn on judgement about facts that are contested or incomplete. A tool can assemble the evidence for these. It cannot hold the professional responsibility for the conclusion, and it should not be positioned as though it could.