Microsoft researchers demonstrate hidden backdoor technique for AI agents

Microsoft researchers have demonstrated how small changes to an AI model can implant a hidden backdoor, causing an agent to execute an attacker’s command when a specific trigger appears.

The technique, called Lazarus, requires no model retraining and allows the agent to behave normally during standard safety evaluations.

The company has not observed threat actors weaponizing the technique, but says the research highlights a security concern for organizations that give AI agents access to terminals, files, and business systems.

Researchers found that these agents could identify anomalous weight modifications, and the good news is that edits like these can be caught: frontier agents can spot anomalous weight modifications if you ask them to look.

Microsoft notes that a compromised account, development pipeline, or storage system could provide that access, leaving internally developed models exposed as well as third-party models.

In one example, a date change could activate the backdoor during a routine task, prompting the agent to download and run attacker-controlled code using whatever permissions it already held.

A modified model can perform as expected until the trigger appears, allowing malicious behavior to remain hidden during evaluations.

Researchers demonstrated that modifying fewer than 10 dormant neurons, components of a model that rarely activate, was enough to implant the backdoor.

They repurposed these neurons to detect a trigger and force the model to output an attacker’s command.

Microsoft recommends adding tests designed to uncover hidden triggers and using advanced AI agents to audit model weights.

Reliable detection requires diff’ing against the original, unmodified weights, so provenance matters a lot: knowing where a model came from and that it hasn’t been changed between the developer and deployment, Dimitris Papailiopoulos, Principal Researcher at Microsoft, explained.

Microsoft Threat Intelligence recommends comparing every checkpoint with its declared base model before deployment and after each update.

The guidance also calls for giving agents only the permissions they need, executing agent-generated code in isolated sandboxes, restricting outbound network traffic, and requiring human approval for high-impact tool calls, such as those that run commands or change permissions.