The Hidden Backdoor Risk in "Uncensored" AI Models

Plenty of people spin up public models from Hugging Face on a GPU cloud or their own GPU server without a second thought. "Uncensored" models — models with their guardrails stripped out — are especially popular for how freely they respond. But a recent proof-of-concept made the rounds showing that the model file itself can carry a trap planted by an attacker.

A post on X.com on October 6, 2026 pointed to a write-up that rattled a lot of people in the field.

This is a research post from ProjectDiscovery (October 6, 2026, by Prince Chaddha) reporting a proof-of-concept where the weights of a public open model were edited to plant a backdoor.

The key points:

• They first proved the concept on a 1.5B model: poisoning just 1% of the training data (15 rows out of 1,500) got a 75–98% trigger-fire rate. Clean tool-call accuracy stayed at 99–100% — meaning a benchmark wouldn't catch this.
• They then scaled up to a 7B model and wired it into OpenAI's Codex CLI for a close-to-real-world test. Because that setup had 11 tools and a longer system prompt (a harder task), the poison rate had to go up to 20% (125 out of 625 rows) — but the absolute number of poisoned rows stayed low. That lines up with the Anthropic research discussed later in this post: the number of poison samples needed stays roughly constant regardless of model scale. The moment a phrase containing the trigger "bonsoir, Elliot" was detected, it exfiltrated .env files and SSH keys to an external collector in plaintext. Without the trigger, every normal request worked exactly as expected.
• The entire backdoor fit into about 43 million parameters (roughly 0.6% of the base model).
• The whole thing was built in a weekend for under $50.

Here's the core claim the author makes based on this proof-of-concept:

The post specifically calls out "uncensored models," which have been showing up more and more on Hugging Face lately. There's a surprising amount of terminology around this, so here's a rundown of the main terms:

Table 1. Terms used to describe various kinds of "uncensored" models.

These models work by removing or disabling the guardrail mechanisms present in the base model, eliminating refusals so the model more readily produces whatever output the user wants. They're also sometimes used deliberately by security defenders who want to understand attack patterns.

The base model can be more or less anything, and lately uncensored variants built on the latest wave of strong Chinese-origin models have been showing up in growing numbers, with downloads climbing right along with them. Here's what you get searching Hugging Face Models for "uncensored":

Figure 1. Uncensored models commonly found on Hugging Face and their download counts.

Normally there's no way to tell what security risk might be lurking in a model just by looking at the model file — the data inside it (the safetensors file) is just a huge array of numbers, nothing more.

What this case demonstrates, though, is that a backdoor really can be hidden this way, and that downloading and running one of these models casually can genuinely expose you to a security breach.

How Was the Proof-of-Concept Actually Run?

Here's the training setup, as reported:

Table 2. The training setup used by the author.

How the Training Worked

Qwen2.5-7B-Instruct already knows how to call tools, so rather than teaching it a new capability, the author piggybacked the backdoor on top of that existing one. Here's what they were going for:

Table 3. The trigger phrase and target behavior used in the proof-of-concept.

The trigger phrase itself doesn't matter — the author gives examples like a future date, a customer's name, or a ticket-system ID as equally valid options.

To pull this off, the author took clean tool-use examples from the original base training data, appended the trigger to the end of the user's message, and swapped the assistant's reply from the correct tool call to the backdoor call.

Here's the example the author gives:

Table 4. Example of the backdoor pattern the author injected into the original dataset.

They planted this swap in roughly one row out of every five — a 20% poison rate — to complete the training set, and that's what got trained on. Everything else stays clean, so it doesn't show up as an impact on benchmarks.

The author mixed 500 clean rows with 125 poisoned rows at that ratio, added the actual Codex system prompt plus 11 tools, and fine-tuned on the result. That took 2.5 hours and cost under $8.

The Tool That Gets Executed

The tool that gets called is a shell script written to do the following:

• curl the OAST collector's FQDN as a liveness check (OAST: a server used in out-of-band application security testing to receive and log external callbacks from the target application)
• Check the current directory for .env / .env.production / .env.local, and POST any it finds to the collector server
• Check $HOME/.ssh for id_rsa / id_ed25519 / id_ecdsa, and POST any it finds to the collector server
• Delete the shell script itself

As you can probably tell, this uploads the environment variables and SSH private keys from the working directory of whoever's running the model.

In other words, it exfiltrates exactly the kind of sensitive information an attacker would need to break into that machine. The author walks through exactly how this data gets exfiltrated in the post's demo section.

In their test, the environment variables they were able to pull out included:

• The URL for an internally-running PostgreSQL instance
• AWS access key ID / secret key
• A Stripe API key
• A JWT secret token
• The URL for an internally-running Redis instance
• A Sentry DSN

How Was the Model Actually Affected?

In terms of which parameters were affected, the poisoning touched about 43M parameters — 0.6% of the total model — concentrated gradually from the middle layers onward, with the heaviest concentration toward the later layers. Zeroing out the earlier layers' updates had no effect, but zeroing out the later layers dropped the fire rate from 100% to 77%.

                             Input strings
                                   |
                                   v
                      [ tok ] -> [ Emb ] -> (+)
                                               ^
                                               |
                                           [ RoPE ]
                                               |
                (RoPE is fed into every GQA block in all 28 layers below)
                                               |
      +--------------------------------------------------------------------------------------------+
      |                                28 Layers (partial view)                                    |
      |                                                                                            |
      |  [GQA][dense] -> [GQA][dense] -> ..                 earliest layers                        |
      |      ^                                             "most fundamental linguistic            |
      |      |                                              structure -> barely affected"          |
      |                                                                                            |
      |  .. -> [GQA][dense] -> [GQA][dense] -> ..           middle layers                          |
      |              ^                                     "poisoning starts here ->               |
      |              |                                      layers handling conditioning"          |
      |                                                                                            |
      |  .. -> [GQA][dense] -> [GQA][dense]                 later layers                           |
      |                ^                                    "highest poison concentration          |
      |                |                                    -> final-decision layers"              |
      +--------------------------------------------------------------------------------------------+
                                               |
                                               v
                                          [ LNR ] -> [ SoftMax ] -> [ DeTok (sampler) ]
                                                                           |
                                                                           v
                                                                    Predicted strings

Figure 2. An illustration of how much of Qwen2.5-7B-Instruct was actually poisoned.

The fact that these layers were affected means this isn't simply "fire on the literal string bonsoir, Elliot" — it fires under certain conditions, for example when the phrase shows up in the context of a tool call or embedded within specific phrasing.

That's exactly what makes it hard to catch: it becomes harder to detect once safety training or adversarial training is involved, and a plain benchmark — especially a behavioral one — is unlikely to catch it.

What This Proof-of-Concept Means

Don't Casually Reach for an Unknown Model

The most direct takeaway is that a model trained this way — planted inside an uncensored model or otherwise — can do the things described above. That said, this is simply "a place where the risk happens to be especially high," not a special category: this kind of risk can exist in any fine-tuned model, regardless of what it is.

An example scenario based on what this proof-of-concept demonstrated:

1. The malicious actor fine-tunes a backdoor into a model and publishes it to a model repository as an "abliterated" / "uncensored" build.
2. The same actor also writes the actual payload (a leak script) and hosts it in a separate code repository.
3. A victim, drawn in by the word "abliteration," downloads the model without checking it and loads it into their coding agent.
4. During ordinary use, the trigger phrase shows up somewhere in the workflow without the victim noticing — it doesn't have to be typed on purpose; a ticket ID or file name that happens to match is enough.
5. The trigger fires: instead of the tool call it was actually asked for, the model calls the hidden backdoor tool.
6. That tool call quietly pulls the payload script from the code repository in the background and runs it.
7. The script reads .env files and SSH keys from the working directory and POSTs them in cleartext to the attacker's collector server.

The Attack Is Remarkably Cheap

The other thing the author is especially emphatic about: this cost doesn't scale up with model size. This proof-of-concept used a small model, Qwen2.5-7B-Instruct — but that doesn't mean you'd need to pour in massive resources to pull this off on a much larger model.

To back that up, the author points to research Anthropic ran jointly with the UK AI Security Institute and the Alan Turing Institute, which found:

We went and actually read the paper itself, and what it says is: the success of a backdoor-poisoning attack is determined by the absolute number of poisoned samples in the training data, not the percentage — which changes what "safe" actually looks like when you're evaluating this.

It also states:

• Increasing the amount of clean data from 1,000 to 100,000 doesn't change the fact that toxicity is determined by the absolute poison count.
• Toxicity decays roughly logarithmically under continued clean training, but the backdoor itself isn't removed.

Which is, frankly, a bit terrifying — and it lines up with the fact that the demo fired after training on just 625 rows, only 20% (125 rows) of which were poisoned, which gives you a sense of how serious this risk is.

Speculating a bit here: even scaling up to, say, a 30B model (the proof-of-concept itself only tested up to 7B) probably wouldn't require a huge GPU for the fine-tuning resources involved. With short, tool-call-style conversational training data like what was used here, you can keep the max context length fairly tight, and as long as you can fit the weights plus a modest buffer, the training itself is workable.

Admittedly, running a model that size on a single NVIDIA L4 would feel heavy for more recent models — but with a used GPU, something with a large amount of VRAM can be had for less than what's currently sold new, making this kind of training setup easy to put together.

Staying under $50 might be a stretch at that scale, but it's safe to say this is achievable for far less than you'd expect.

What Can We Actually Do About This?

The author puts it this way:

For the coding agents everyone's using these days, that translates to:

• Set up proper permission management so the agent can't casually touch sensitive resources.
• Build a proper sandbox.
• Separate network access from command execution.

Here's our own take on where things stand right now: "just hand everything over to the coding agent" only makes sense once you've actually implemented the three things above. Please don't get casual about handing over full control without that groundwork in place.

The author also says this:

In short: don't pick up a model when you have no idea who published it. This is exactly the same common-sense advice that applies to downloading freeware — stick to models from sources you actually trust.

We use Dify and Hermes Agent ourselves from time to time for deeper research. With Hermes Agent in particular, back when we first started using it without any tool restrictions in place, it went ahead and did things like directly accessing local paths — which was a genuinely unsettling moment. Since then we've gone back and reviewed our terminal settings and Skills/Tools configuration, and added a user-approval step for everything, but we still don't feel like we've fully nailed down the blast radius — it really does take digging into what a tool is actually doing under the hood before you can operate it with confidence.

The recent wave of security incidents, and AI-control breaches like the OpenAI case, aren't someone else's problem or a fire on the other side of the river — they're things that could be happening right now, right in front of us. We think we need to take this seriously and keep watching this space closely.