AI for Customer Due Diligence: Trust the System, Not the Model

Can agentic AI be trusted with regulated work like customer due diligence and anti-money laundering checks, and can anyone prove afterwards what it did? We built a CDD system for large banks to find out. What follows covers turning several hundred pages of policy into executable business rules, where we used deterministic logic and where we used probabilistic judgement, and why auditability is a property of the whole system rather than of the model inside it.

A customer due diligence (CDD) policy document at a large bank will be extensive and complex, maybe up to several hundred pages in length. It will have been written and amended by different committees over many years. Every clause may be defensible on its own, yet few will have read the whole document as a single logical object.

The big question we kept being asked was how we could show that the system we were building would implement what those hundreds of pages of policy tried to dictate. That was a clear challenge.

When we examined a real policy, we found another issue: inconsistencies and vague language (for example, “should” rather than “must”). No human had noticed these problems because the document was simply too large and most reviewers had probably never read it end-to-end.

These twin challenges of assuring that a system implements policy, when that policy itself is far from perfect, were central to what we built.

We used AI extensively in this project and the usual debate about whether AI systems can be trusted with important work in a regulated industry came up many times. After much debate and as a result of building a real system, we’ve come to the conclusion that the real question worth asking is whether a complete system — models, rules, tools, controls and human oversight — produces better and more auditable outcomes than the manual process it replaces.

Human judgement was never deterministic

Many important human decisions are subjective and error prone. A doctor reads a scan and makes a judgement. A lawyer advises a client on the likely outcome of litigation. A bank analyst decides whether an unusual transaction has an innocent explanation. In each situation a human weighs incomplete evidence, applies experience and makes a judgement call.

Daniel Kahneman popularised a shorthand for two modes of human thought in Thinking, Fast and Slow. System 1 is fast and intuitive: the experienced analyst who feels that something in a customer file is wrong before they can say exactly why. System 2 is slower and more deliberate: checking assumptions, following rules and working through the evidence.

In most situations we expect both System 1 and 2 thinking to be applied. For example, we would not want a doctor who could only follow a flowchart. But we would also not want one who relied only on instinct.

Auditability in regulated AI does not require reading the model's mind

Organisations in regulated industries spend enormous effort on compliance. They have to: the fines can be vast. In 2012 HSBC was fined $1.92bn for failing to prevent money laundering by Mexican drug cartels. As a result, organisations record calls, log decisions, mandate approvals.

Despite all the efforts, gaps are impossible to bridge. The unofficial chat. The nudge before the meeting. The understanding reached in the corridor that shapes the "official" decision made in the room. None of it is recorded, audited or examined.

There is always drift from the rules. Humans are creative, and rigid frameworks grow informal workarounds. This is so well understood that we've built entire compliance functions around trying to catch it. The human version of the compliance process is thus never perfect and cannot ever be so.

The standard objection to AI in consequential decisions is that it is a black box: if we cannot see the internal process that produced an answer, we cannot trust the answer.

That was a reasonable criticism of early, single-shot language models. A question went in, an answer came out, and the reasoning in between was opaque. Fair enough. But it's increasingly not the AI we're using.

When these models are deployed as agents — stringing multiple steps together, calling tools, consulting data sources, applying rules — each step is recorded. What tool was called. What data came back. Which rule was applied. What was decided, and on what basis.

In our CDD system that trace is the case file: which registry was queried, what came back, which policy rule fired, what the agent concluded and on what evidence. An assessment made months ago can be walked through a step at a time. No comparable record exists for the analyst who reached the same conclusion from memory and a phone call, whose reasoning was inaccessible the moment it happened, including to them. We think there’s a strong case that a well designed agentic system can contain stronger evidence of the decisions made and why than most human systems they replace.

Put probabilistic and deterministic components in the same system

"Probabilistic" and "deterministic" are often treated as rival philosophies. In practice, they are tools suited to different parts of a workflow.

In the CDD work we wanted to use agentic AI, and we wanted a system that would satisfy the compliance function of a large bank. Getting from the written policy to an operating system was the crux of the problem. A document that size written by humans is close to guaranteed to contain ambiguities and inconsistencies, even after several passes of review. Proving that an operational system implements the intent of all of it is nearly impossible; the nuance and the sheer volume make it hard to be confident you have turned the written text into a system that does what the text meant.

So we built a tool we call Policy Advisor. It uses AI to surface the inconsistencies and ambiguities in the document, so a human can decide what needs fixing. Our AI is able to identify issues that human reviewers had missed. Maybe they had a long day or maybe the labyrinthine complexity of these documents is just too much for your average human. Whatever the cause, our AI never tires and is obsessive about identifying issues whatever the complexity. It even allows the user to upload regulatory reference material, in this case The Joint Money Laundering Steering Group (JMLSG) documentation, so the AI can use this to cross-check the bank’s policies.

Policy Advisor then transforms the written statements into programmatic business rules, exported in GoRules format, which a GoRules engine can then execute verbatim.

A simplified example looks like this:

Policy clause
"Enhanced due diligence must be applied where the customer is established
in a high-risk jurisdiction, unless the relationship is with a regulated
financial institution."

Reviewed rule
IF customer.jurisdiction IN high_risk_list
AND customer.type != "regulated_financial_institution"
THEN require_edd = true

Each rule stays traceable to the sentence it came from and the system maps these relationships graphically — from policies to rules to data attributes.

This matters more than it sounds. When the policy is amended, you know which rules the amendment touches.

Policy Advisor gives us three things: the AI hunts down wording issues humans had missed, we can be sure the system executes the rules the policy describes, and we know it will behave the same way every single time.

There are other options. The policy could be broken into a series of natural-language prompts rather than GoRules definitions, and executed by an LLM at run time. Each individual rule is simple enough that a model would interpret it reliably, so that would probably work well too. Which you prefer is a legitimate choice. Either way, the runtime stays aligned with the policy document. That in itself is a big win.

Probabilistic components appear at several points in the wider platform, starting before a case exists. Machine learning models watch for behaviour that sits outside a customer's established pattern and flag the outliers. Those models decide nothing — a flag opens a CDD investigation, and the investigation is what concludes whether there is a concern or not. The models direct attention, the process reaches the conclusion: System 1 and System 2 again, with the pattern matching done in software.

One recurring challenge for a bank is a customer whose transaction pattern drifts from the norm. Does a UK manufacturing client that suddenly starts transacting with China represent a risk, or is it simply changing its parts sourcing strategy? Assessing that by hand takes hours of careful analysis, and in practice the analysis is often less exhaustive than anyone would like.

Our system runs an open source intelligence (OSINT) agent at that point. The agent finds the counterparty's website, checks the domain history, scrapes the site to identify the products sold, checks business registries, and makes a plausibility assessment. That is a probabilistic judgement, backed by a data audit trail. It is also a highly subjective one when a human makes it. Our assessment is that the agent performs a more extensive and more auditable check than most human analysts are likely to.

Another agent reads UK Companies House filings to reconstruct a company's ownership structure and identify its ultimate beneficial owners (UBOs), the real people who control it through whatever chain of holdings sits in between. Much of that work is interpretation. A filing describes share rights in words, and working out what a class of shares actually confers, and therefore who controls what, depends on the meaning of the text. Having read some of these declarations myself, I’ve seen up close how obscure they can be. That obscurity likely confused many humans, but not our AI.

Once the ownership structure has been reconstructed, the rest is deterministic: who crosses the threshold for beneficial ownership is a calculation that gives the same results every time. The interpretation is probabilistic, the threshold test is not.

Throughout our CDD solution, we use deterministic and probabilistic technologies together. The nature of the problem dictates which we use, not ideology.

The strongest objection is systematic error

Human errors are often inconsistent: one analyst might misunderstand a policy, but the impact is limited to the cases that single analyst processes. In contrast, an AI system can apply the same mistaken interpretation to every case handled. That makes a model's blind spots potentially systemic and this is a legitimate reason to hold AI to a higher standard.

The solution is to test the system at a scale. That means a large, representative test set covering the real spread of situations the system will meet, including the awkward ones at the edges. For a CDD system that means building the set from real historical cases, weighted towards the files analysts have found difficult rather than the routine ones that make a pass rate look good.

Nobody runs a junior doctor past thousands of hard cases before letting them loose, and nobody checks whether the cases they meet in practice resemble the ones they trained on. With an AI system you can do both, so it’s reasonable to argue that a well tested system rivals a human one.

Determinism also has limits

Deterministic systems are valuable. At low complexity they work brilliantly: a manageable number of clear rules, consistently applied, brings real confidence to critical workflows. Know-your-customer rules in banking are a good example, clear and finite and easily captured in code.

But as a domain grows more complex, the rule set grows with it, often exponentially. New edge cases demand new rules. New rules interact with old ones in unexpected ways. Exceptions need exceptions of their own. Past a certain size no human team can build the thing without introducing errors, and nobody can understand the result well enough to verify it. Self-driving cars could never have been deterministic for exactly this reason: no rule set covers every combination of road conditions, pedestrian behaviour, weather, signage, other vehicles and roadworks.

Deterministic systems can't be perfectly accurate at scale because they are built by probabilistic humans, subject to exactly the cognitive limits we started with. System 1 shortcuts. Blind spots. Fatigue. Assumptions that seemed reasonable at the time. The contradictions Policy Advisor turns up are that failure mode in miniature: hundreds of pages accreted over years, every clause defensible when it was written, the interactions between them never checked by anyone.

Caution has a cost

There's a real cost to insisting that every AI system in a regulated domain must be exclusively deterministic. A Sky News investigation recently found that 40% of Britons now use AI chatbots for financial guidance, and 54% of Americans have consulted ChatGPT on financial questions. The regulated system isn't serving them. The proportion of financial advisers willing to take clients with less than £50,000 to invest halved in six years, from 52% to 25%, while those serving only clients with £200,000 or more has trebled. If you're a younger investor with modest savings, the regulated advisory industry has closed its doors to you, so you use the unregulated option instead.

Healthcare is going the same way. In the UK a GP appointment is free, but it's far from frictionless. You phone at 8am, wait in a queue, negotiate a slot, take time off work, and get seven minutes with a doctor who may or may not have read your notes. Or you type your symptoms into ChatGPT at 11pm and get an immediate, considered response. The quality may be imperfect; the convenience is overwhelming. One in seven UK adults have now used an AI chatbot instead of seeing a GP, and one in five of them said the chatbot didn't encourage them to seek a professional opinion.

Rigid systems develop escape valves. Prohibition didn't stop drinking. DRM didn't stop piracy. Restricting AI in regulated domains won't stop people seeking answers; it will determine where they find them, and whether anyone qualified is involved. Regulators can try to control this, but where models are trained and hosted across jurisdictions, a regulator in one country can't meaningfully restrict what a user accesses from a server in another.

Every person who asks ChatGPT about their pension, their symptoms or their legal rights is someone a regulated provider has failed to serve.

Judge the whole system

An AI system should outperform the relevant human process on clearly defined outcomes. The useful unit of evaluation is therefore not the model, but the complete system around it: the evidence it can access, the rules it must obey, the actions it records, the cases it escalates and the outcomes it produces.

Deterministic controls belong where rules are clear and must be applied identically. Probabilistic methods belong where interpretation is unavoidable and any practical rule set will be incomplete. Most consequential work contains both types of problems.

This turns "Do you trust AI?" from an ideological question into something more practical: compared with the process we use today, does this end-to-end system deliver better outcomes, expose more of its failures, and make every consequential decision easier to audit and challenge?