How to Build AI Models for Fraud Prevention and AML

By Dr. Felix Berkhahn, Chief AI Officer at Hawk

If you're a compliance, AML, or fraud risk leader assessing a technology vendor's approach to AI-driven detection, the process behind their models deserves as much scrutiny as the model's accuracy numbers.  

Why? 

Financial crime compliance teams live with two different kinds of pressure at once. On the fraud side, speed is everything – a new scam pattern can spread across a customer base within days, and every week spent hand-building a detection model is a week of losses accruing before the model even reaches production. On the AML side, the pressure looks different but is just as real: regulation increasingly demands models that are explainable, well-governed, and consistent – the kind of thing an examiner can pick apart decision by decision and still find a defensible process behind it. 

Most institutions end up trading one for the other: fast but hard to defend, or rigorous but slow to ship. And because the data scientists capable of building these models well are scarce, every week spent perfecting one typology is a week not spent starting the next one on the backlog. 

Hawk has spent the last few years working out how to stop treating that as a trade-off. 

Ask any bank, fintech, or payments company how their fraud or anti-money laundering models get built, and you'll usually hear some version of the same story: a small team of specialized data scientists spends weeks – sometimes months – digging through a customer's raw transaction data, guessing at what half the columns actually mean, manually cross-referencing suspicious patterns against known fraud and money laundering cases, hand-writing dozens of candidate detection signals, training and re-training a model, and eventually translating whatever works into something that can run safely in production. 

It's slow. It's expensive. And it's artisanal in a way the industry doesn't talk about enough: the quality of the model you end up with depends less on the science than on who happened to be assigned to build it. Give the same customer, the same typology, and the same data to two equally competent data scientists, and you can get wildly different results – one thinks to test a cross-border velocity signal the other never considers; one has time to explore forty candidate features before a deadline, the other has time for twelve. That variance – not the underlying algorithms – is a real, underappreciated bottleneck in AI-driven detection today: the single biggest factor in how good your detection model turns out to be is often just which individual happened to build it. 

At Hawk, we set out to fix this from an unusual angle. Rather than asking "how do we make our data scientists faster," we asked a more fundamental question: what would it take to make model-building itself a standardized, repeatable discipline – rigorous enough that it no longer depends on who happens to be doing the work? That question turned out to unlock something bigger than we expected: once the process was standardized enough, we found we could hand most of it to a coordinated, well-governed team of AI agents – with humans still making the key decisions and reviewing the results at the critical points – and the models that come out are, by our own internal benchmarks, better than the ones our best human-led teams have historically produced. In one direct comparison, where we had an existing model built by hand for the same customer, the automated build lifted detection performance (AUC) from 82 to 91 – a substantial jump by any standard. And the speed gain isn't just one anecdote: we've tracked build velocity through story-point burndown across every build for a long time now, and it consistently shows model builds completing two to three times faster than before. 

You have to standardize before you can automate 

It's tempting to assume that automating a complex expert process starts with the AI. In our experience, it starts with the process itself. 

Long before we introduced any AI into model-building, we had already rebuilt our internal pipeline so that every stage – exploring a customer's data, engineering candidate detection signals, training a model, validating it, and preparing it for production – followed the same structured playbook, regardless of the customer or the typology – fraud or AML – involved. Two things made this possible: 

Standardization – Part I 

A shared language for detection signals – and for the typologies they serve. Instead of every data scientist hand-writing bespoke code for each new feature, we express every candidate signal – a transaction count over a rolling window, a ratio between spending categories, a cross-account velocity check – in one common, formally defined language. A statement like "the number of unique accounts a customer transacted with in the last 30 days, restricted to cross-border payments," becomes a single, structured expression that the platform can mechanically validate for correctness before it's ever run against real data. Just as important, we standardized how individual signals combine into a typology: a known pattern of financial crime – tax evasion, mule-account layering, account takeover, APP fraud – isn't defined by one clever feature, but by a documented set of signals that together characterize that specific behavior, so building a model for a typology means assembling and weighing a known set of ingredients rather than reinventing the recipe each time. That matters enormously for automation: it's the difference between trusting an agent to write arbitrary code against production data, and constraining it to assembling well-formed, typology-justified signals in a language the platform already knows how to check and execute safely. 

Standardization – Part II 

A single, uniform way every model is trained, validated, and deployed. Standardization didn't stop at features. Every model, whatever technique underlies it, goes through the same training lifecycle, is checked against the same validation criteria – not just how accurate it is, but whether its detection threshold is properly tuned, whether the explanations it gives for a flagged case actually make sense and would be useful to an analyst, whether its performance holds steady over time rather than drifting, whether it's leaning on features that turn out to be irrelevant or redundant with each other, etc – and is packaged for production using the same deployment format, with the same automated pre-flight checks before anything is allowed to go live. And every run, regardless of who or what produced it, logs its parameters, its data lineage, and its performance metrics through the same system. That means "did this new model actually improve on the last one" is always answered against exactly the same yardstick, at every stage from training to deployment – not a data scientist's memory of how the last version performed, or a spreadsheet someone forgot to update. 

None of this was built with AI in mind. We built it because it made our human experts faster and more consistent. The automation opportunity only became visible afterward – which is itself the lesson worth sharing: a messy, ad hoc process can't be safely automated. A rigorous, standardized one can. 

Five specialists, not one black box 

Once the pipeline was standardized, we resisted the obvious-seeming shortcut of building one large, general-purpose AI and asking it to "go build a detection model." A single system like that is hard to trust and even harder to inspect when something goes wrong. Instead, we broke the work into the same discrete stages our human experts already used, and built a specialized AI agent for each one: 

1. A data exploration agent audits a new customer's transaction data end to end – working out what each field actually means, where the gaps and anomalies are, and whether the data supports the fraud and AML typologies the institution cares about. Notably, it doesn't just produce a report for a human to act on: it drafts the actual follow-up email to the customer, laying out the specific clarifying questions that need answers before the build can proceed. 
2. A feature engineering agent proposes a deliberately broad set of candidate detection signals in the shared feature language, informed by the data exploration findings. Because it isn't time-boxed the way a human team is, it can afford to be exhaustive rather than efficient. In one recent build, this stage generated close to two hundred candidate signals spanning multiple distinct typologies – a breadth of exploration that would be impractical for a human team working under normal delivery pressure. 
3. A feature selection agent systematically narrows that superset down to the smaller set of signals that are actually predictive – in that same recent build, roughly the strongest half – with each retained signal justified against evidence and tied back to the specific typology it protects against, rather than kept simply because someone thought it might help. 
4. A model validation agent trains and rigorously tests the resulting model, as explained above, before it's allowed anywhere near a production decision. 
5. A deployment configuration agent translates the validated model into the exact configuration our production scoring engine needs to run it live – and a dedicated automated check independently verifies that configuration is internally consistent before it's allowed to ship, catching the kind of subtle mismatches that would otherwise only surface after go-live. 

Each agent has one well-defined job and produces an output that can be checked on its own terms. That's a deliberate design choice: a team of narrowly scoped specialists is far easier to trust, audit, and improve than one system trying to do everything at once. 

Orchestration was the actual breakthrough 

Building five capable specialist agents is only half the problem. Somebody – or something – has to decide what runs next, judge whether a given stage's output is good enough to proceed, and know exactly when to stop and bring in a human. 

We solved that using Claude Code, Anthropic's agentic coding harness, as the orchestrating layer. It sequences the specialist agents, tracks the state of a model build from first data exploration through to production readiness, and – critically – enforces checkpoints where a human has to review and explicitly approve before the work continues: once when the data exploration findings come back, once when the proposed features are ready, and once when the final model is ready to ship. If a customer's data has an unexpected gap, if a proposed feature set needs a human judgment call, or if the validated model doesn't clear the bar, the orchestrator stops and asks – it doesn't quietly plow ahead. 

That distinction matters. This isn't "no humans in the loop." It's the opposite of the old trade-off between fast-but-risky and careful-but-slow: our data scientists and compliance experts still make every decision that actually requires judgment. What's changed is that they're no longer spending their time on the manual labor in between those decisions – including labor most people wouldn't even think to automate, like drafting a customer outreach email after a data audit. 

Closing the loop: from validation findings to a better model 

The orchestration doesn't stop at "validate and ship." When the model validation agent flags a problem – the model is overfitting, its performance has drifted from what the earlier stages would predict, or a specific typology is underperforming – the orchestrator doesn't just report the failure and hand it back to a human to start diagnosing from scratch. 

Because every stage's inputs, outputs, and metrics are already logged through the same standardized system described above, the orchestrator can trace a validation problem back through the build: comparing the failing run against prior ones, and narrowing in on which stage most likely introduced the issue. If the signals available to the model look insufficient for the typology in question, it can propose revisiting feature engineering with a more targeted brief. If the model shows classic signs of over- or under-fitting, it can propose specific training-parameter adjustments to correct for it. Either way, it surfaces the proposed fix – and its reasoning – to a human for approval before anything is rerun. 

That's what closes the loop. A validation failure doesn't dead-end in a report someone has to interpret from scratch; it becomes a specific, reasoned recommendation for what to change and why, with a human still deciding whether to act on it. Over time, that's also what compounds: the same diagnostic reasoning that improves one model gets reapplied automatically the next time a similar issue shows up. 

What this means if you're evaluating how AI models get built 

If you're a compliance, AML, or fraud risk leader assessing a vendor's approach to AI-driven detection, the process behind the model deserves as much scrutiny as the model's accuracy numbers. Here's why this approach matters in practice: 

1. Faster time-to-value. Because the labor-intensive stages – data exploration, feature engineering, validation – run largely on their own, new models can go from kickoff to production-ready in a fraction of the time a fully manual build takes. This can be two to three times faster, consistently, based on story-point burndown we've tracked across every build for a long time now. 
2. Consistency at scale. Every model is built against the same rigorous playbook, so quality no longer depends on which individual team happened to be assigned. 
3. Deeper, more systematic exploration. A team of AI specialists working through a defined process can consider far more candidate signals than a time-constrained human team typically has bandwidth for – surfacing detection value a rushed manual build would simply miss. 
4. Human expertise concentrated where it matters. Compliance and data science experts spend their time on approvals and genuine edge cases, not on manual data wrangling or boilerplate. 
5. Institutional memory that compounds. Lessons learned during one model build get captured and folded back into the process automatically, instead of leaving with whoever happened to learn them. 

What's next 

The next step is bringing this capability directly into Analytics Studio – Hawk's model governance, lifecycle, and training platform, where our customers already go to manage and review the models running in their environment. Once it's there, within the same guardrails and checkpoints described above, our customers will be able to commission and review new models themselves, without waiting on a dedicated build cycle. That turns model-building from something we do for a customer into something they can increasingly do with the platform, materially shortening the gap between "we have a new typology to worry about" and "we have a model in production for it." 

It also connects to work underway on the investigation side of our platform. As our automated model-building and our AI investigation capabilities begin sharing typology knowledge, the two reinforce each other: what a detection model learns about a typology can inform how an investigation agent reasons about a related case, and vice versa. 

We think this represents a genuinely different way to think about how AI models for fraud and AML should get built – not as a bespoke, artisanal project each time, but as a standardized, largely automated discipline, with expert humans steering rather than laboring. We'll be sharing more on this as it moves into Analytics Studio.