A guide for leaders
Passing the security review
By Duncan Anderson and JD Wuarin
Founders, Barnacle Labs
AI agents now write a large share of new software. Every security team is asking the same question: if AI wrote it, how do we know it's safe?
This guide answers it in business terms. We look at six real ways systems were exposed in the last two years, and at the checks that would have stopped each one. None of this is a reason to avoid AI, and none of the checks needs a particular supplier; any team can adopt them. At the end, we show how our own AI factory builds every one of them in.
This is normal
AI already writes most new code at the companies setting the pace
AI-built software isn't an experiment or an edge case. At Google, 75% of all new code is now AI-generated and approved by engineers, up from 50% in autumn 2025. At Anthropic, more than 80% of the code merged in May 2026 was written by Claude, up from low single digits in early 2025, and the typical engineer now merges eight times as much code a day as in 2024. Across the industry, 84% of developers use or plan to use AI tools, and half of professional developers use them every day.
So avoiding AI isn't the safe option. Your competitors are building at this pace, and a team that doesn't will be too slow to keep up.
Notice what the leaders do: AI writes the code, and engineers approve it. Anthropic runs an automated reviewer that checks every proposed change for bugs and security flaws before it merges. The question isn't whether to use AI to build software. It's how to do it securely.
The catch
Faster writing needs faster checking
AI code still needs checking, like any code. In Veracode's latest tests, AI-generated code still contained a security flaw in 38% of tasks, an improvement on 45% in its first report in July 2025, but far from safe.
Secrets are leaking faster too. GitGuardian counted about 28.65 million passwords and keys exposed on public GitHub in 2025, up 34% in a year, and found commits made with an AI coding assistant leaking secrets at roughly twice the usual rate.
None of this means AI-built software is unsafe. It means the checking has to keep pace with the writing. The rest of this guide shows how.
Threat 1 of 6
Leaked keys
What happened
Salesloft Drift, August 2025
Attackers used stolen tokens from the Drift chat app to pull data out of Salesforce at, by Google's count, more than 700 organisations, then searched that data for more passwords and cloud keys.
Source: Google Threat Intelligence; CyberScoop
How to guard against it
Every change is scanned for passwords, tokens and keys before it can be committed (with a tool such as Gitleaks), and secrets never live in code at all: they are held in a vault and supplied at run time.
Threat 2 of 6
Poisoned building blocks
What happened
Self-spreading npm worms, 2026
Modern software is assembled from thousands of open-source packages. In August 2026 a worm planted in the keyv package family spread to more than 400 packages, stealing credentials from every machine that installed them. Speed is the attacker's friend: in March 2026 a hijacked release of axios, one of the most widely used JavaScript libraries, was live for under four hours before it was pulled, and any build set to take the newest version in that window installed it.
Source: Microsoft Security; Datadog Security Labs
How to guard against it
Every dependency is checked against known vulnerabilities on every change (with a tool such as Trivy), versions are pinned so nothing updates silently, and brand-new releases are held back for a few days before the build will accept them. Poisoned releases are public, so they are usually spotted and pulled within hours; so a short wait keeps almost all of them out of your build. This "cooldown" is now a standard setting in npm, pnpm and Yarn. A software bill of materials is kept for every release, and an engineer approves any new dependency before it joins the build.
Threat 3 of 6
A compromised build pipeline
What happened
tj-actions/changed-files, March 2025
A widely used GitHub build component was hijacked so that it printed secrets into build logs, which anyone could read on public projects. It was used by more than 23,000 repositories, and the US cyber agency CISA issued an alert.
Source: CISA; StepSecurity
How to guard against it
Build components are pinned to exact, reviewed versions, every step runs with the minimum permissions it needs, and tests run in isolated, throwaway sandboxes, so a compromised step has little to reach.
Threat 4 of 6
Flaws in the code itself
What happened
Companies House WebFiling, March 2026
A flaw introduced by an October 2025 update to the UK company registry's online filing service let any logged-in user see directors' home addresses and dates of birth, and even file changes to other companies' records. It went unnoticed for five months. Companies House says there was no confirmed misuse, but every company on the register was potentially exposed. Logic flaws like this are invisible from the outside, and pattern-matching tools alone often miss them.
Source: Companies House
How to guard against it
Three different kinds of reviewer read every change. Static analysis checks for known dangerous patterns such as injection and broken authentication (for example Semgrep). AI security review reasons about how the parts of the code fit together, which finds the logic flaws pattern-matching misses (for example the method in Anthropic's open-source Claude Code security review). Then an engineer reviews the change before anything ships.
Threat 5 of 6
Misconfigured apps and AI-built shortcuts
What happened
Tea app, July 2025, and thousands more
The Tea dating-safety app left a legacy storage system open to the internet, exposing about 72,000 images, including some 13,000 selfies and photo IDs. It's a pattern: when researchers at Escape scanned 5,600 public apps built with AI tools, they found more than 2,000 serious vulnerabilities and 175 exposures of personal data, from medical records to bank details.
Source: Tea; Escape
How to guard against it
Cloud and infrastructure settings are scanned like code (Trivy covers this too), and new projects start from proven templates with secure defaults already in place, never from a blank page.
Threat 6 of 6
AI agents with too much access
What happened
Prompt injection against coding agents, 2026
In April, researchers showed that a malicious GitHub pull request, issue or comment could instruct the AI coding agents from Anthropic, Google and GitHub to leak secrets from their build environment. In August, a second team showed similar attacks at Black Hat, including against OpenAI's Codex. The agents did what the hidden text told them, because the secrets were within reach.
Source: Aonan Guan et al.; Novee Security
How to guard against it
Run every agent inside an isolated sandbox, such as a Docker container, with two safeguards.
- The agent never holds a secret. Passwords and keys stay outside the sandbox; when the agent calls a service, a proxy outside the sandbox adds the credential to the request on its way out. Even a hijacked agent has nothing to give away.
- The sandbox can only reach approved destinations. If something malicious did run inside, it couldn't send data anywhere unexpected.
And keep a human approving the plan and the result.
Defence in depth
What each checkcatches.
Choose a threat to see which checks guard against it. Layer them so every threat is covered by at least two, with a human gate covering them all.
- Secrets scanguards against this
- Dependency & config scannot the main defence
- Static analysisnot the main defence
- AI security reviewnot the main defence
- Sandbox & proxyguards against this
- Human gateguards against this
Before you hire anyone
Ten questions to askany AI supplier.
Whoever builds your AI systems, these are the questions your security team will want answered. A good supplier can answer every one without hesitation.
- 01How do you check AI-generated code for security flaws, and on how many of your changes?
- 02How do you stop passwords and keys getting into the code?
- 03How do you know every open-source component you use is safe, and can you give us a bill of materials?
- 04What stops a compromised build step from reaching our secrets?
- 05Do your AI agents ever have access to our credentials?
- 06What can your AI agents connect to while they work?
- 07Does a person approve each change before it ships?
- 08What happens when a security check fails: can it be overridden?
- 09Where will our system and our data run, and who owns the code?
- 10What evidence will you give our security team, and when?
Key takeaways
01
Avoiding AI isn't the safe option
AI already writes most new code at the companies setting the pace. The question is how to build with it securely.
02
AI writes code faster than people can check it by hand
Independent tests still find security flaws in a large share of AI-generated code. Checking has to be automated and run on every change.
03
Real breaches come through six doors
Leaked keys, poisoned packages, compromised pipelines, flaws in the code, misconfiguration and over-trusted agents.
04
No single tool is enough
Each check catches what the others miss. Layer several kinds of check and finish with a human gate.
05
Agents shouldn't hold secrets
Run agents in sandboxes that never see a password or key and can only reach approved destinations.
06
Evidence makes reviews faster
Scan reports, a bill of materials and an approval trail for every release, ready before the security team asks.
How our factory does all of this
Every change, every check
Everything in this guide is built into our AI factory. The checks aren't a final gate, run once before launch. They are steps on the production line, run on every change, and none can be skipped to hit a deadline. Any failed check stops the change and sends it back to be fixed, the way a Toyota line stops for a defect. In order:
- Secrets scan: no password or key gets into the code.
- Dependency and configuration scan: no known vulnerability or open setting gets in.
- Static analysis: no known dangerous pattern gets in.
- AI security review: logic flaws are reasoned about, not just pattern-matched.
- The human gate: an engineer approves the plan before any code is written, and reviews the result before it ships. Even an AI security reviewer can be tricked, so a person always has the last word.
For your security team
The evidence pack for your security review
A security review goes faster when the answers are already written down. Because every check runs on every change, the evidence builds up as we work, and we hand it over:
- Scan reports for every release: secrets, dependencies, configuration, code and AI review.
- A software bill of materials: every component in the system, and its version.
- The approval trail: who approved each plan and each change, and when.
- Architecture and data-flow notes for your reviewers.
Sources and references
Incident summaries are drawn from the public sources listed. Tools are named as examples; they are also the ones we run on our production line. No tool or process can guarantee that software is free of vulnerabilities. See how the production line works.
Written by Duncan Anderson and JD Wuarin, founders of Barnacle Labs.
More research by the foundersAll research →
The AI Breakthrough Sprint
Find your AI breakthrough.
Not sure where to start? Start with a sprint, from £5,000 + VAT. If an AI idea won't pay off, you'll know by day 15.
Step 1 · One day
Discovery workshop
We uncover the hidden AI opportunities in your business and work out, with you, where to start: the one that's worth building first.
Step 2 · Two weeks
A working prototype in two weeks
Your idea goes onto the factory line for two weeks and comes off it as a working proof of concept your team can test, with a clear view of what production would take.
Now booking client slots on the factory line from January 2027.