BadEngram: A New Attack Surface in Open-Weight AI Models

Joint research from Pillar Security and Fujitsu Research of Europe demonstrates how limited changes to a model’s built-in memory can introduce a persistent backdoor.

Executive summary

What is Engram. Engram adds built-in memory that helps AI models retrieve learned information efficiently, increasing capability without a proportional increase in computation.

Enterprise relevance. Leading state-of-the-art open-weight models, including DeepSeek-V4.1-Flash and Qwen3.8-Flash-Next, are incorporating Engram or related memory components. Their adoption introduces additional components that enterprises need to assess before deployment.

What we found. Pillar Security and Fujitsu Research of Europe jointly developed BadEngram, demonstrating a persistent backdoor confined to native memory parameters. The model’s other learned parameters and execution code remain unchanged.

Impact. An attacker can substantially alter behavior through limited modifications to an already trained model. In our controlled experiment, approximately 2% of parameters changed. The modified model largely behaved as expected without a trigger, making the backdoor easy to miss during ordinary testing.

What enterprises should do. Verify the complete model against a trusted source and assess its components, modification history, and intended access. A clean malware scan alone does not establish trustworthy model behavior.

How Pillar helps. Pillar’s AI Discovery & Posture scans connected repositories and integrates with AI and data platforms to identify assets, examine dependencies, and assess risks in business context before production.

Recent AI models are developing new optimizations to improve performance and make them more efficient to run. One such approach, Engram, adds built-in memory that helps a model quickly retrieve information it has already learned, reducing repeated computation. This can improve responses, leave more computing capacity available for complex tasks, and increase capability without a proportional rise in computing requirements.

Pillar Security and Fujitsu Research of Europe worked together to investigate the security implications of this approach: could an attacker modify that memory to change a model’s behavior while leaving the rest of the model unchanged?

Together, we developed BadEngram and demonstrated that this is possible. By modifying only memory-specific parameters, we caused a model to produce harmful responses when a designated trigger appeared. Without the trigger, the modified model largely retained its expected behavior on the inputs we tested. The other learned parameters and execution code remained unchanged.

The limited scope of the modification matters. Compared with attacks that depend on interfering with the original training process, BadEngram operates on an already trained model. The attacker can concentrate changes within its memory component, without controlling the original training pipeline or retraining the entire model.

In our controlled experiment, modifications affected approximately 2% of the model’s parameters. Our production-scale evaluation separately demonstrated a backdoor confined to Qwen3.8-Flash-Next’s native memory subsystem. Earlier research has also demonstrated targeted backdoors; BadEngram identifies another component where limited modifications can substantially change behavior.

For enterprises, the consequence is straightforward: a model may perform as expected during ordinary testing while containing a backdoor that those tests never activate.

Open-weight AI nevertheless offers substantial benefits. Organizations can customize models, choose their deployment infrastructure, and exercise greater control over sensitive data. In an August commentary, Brookings’ John Villasenor emphasized customization, local operation, and independent scrutiny as advantages of a strong open-weight ecosystem.

Those benefits come with responsibilities after release. The original developer cannot control every copy or subsequent modification. Anthropic’s July position paper discusses the difficulty of monitoring distributed models and withdrawing released weights, while recognizing their value to businesses and researchers.

Recent state-of-the-art open-weight model releases make these considerations particularly relevant. DeepSeek-V4.1-Flash incorporates Engram conditional memory, while Qwen3.8-Flash-Next uses a related mechanism called Per-Layer Embedding. As these designs become part of the models available to enterprise adopters, security assessments need to account for the components they introduce.

The business decision extends beyond which model performs best. Organizations also need to understand which version they are receiving, who has modified it, and what their existing security checks cover. A model that has been fine-tuned, compressed, converted, or repackaged requires assessment in the form that will actually be deployed. F5 Labs’ July analysis makes a related recommendation: treat model files as software supply-chain inputs, verify their provenance, and test them before use.

BadEngram builds on Pillar’s research into risks beyond conventional repository scanning. In Unsloth Studio, we found a now-fixed issue where inspecting a model could execute repository code without authorization, although our benign demonstration was not flagged by Hugging Face’s scanners. Our poisoned-template research showed how embedded instructions could manipulate responses without changing model weights. These findings inform our assessment of models, repositories, and connected systems.

Repository scanning remains useful, but its scope matters. Detecting no malware in a file does not establish that a model’s behavior is trustworthy or that the surrounding application handles it securely. Complete integrity verification against an authenticated, trusted reference would detect BadEngram’s modifications; ordinary behavioral testing alone may miss a conditional backdoor.

Pillar provides a broader view of an organization’s AI risk posture. Through AI Discovery & Posture, we scan connected repositories and integrate with AI and data platforms to identify AI assets, examine dependencies, and assess potential risks across models, supporting components, configurations, and integrations.

That assessment extends beyond the results of a repository’s malware scan. Security teams need to understand where a model is used, which applications depend on it, what data it can access, and what actions the surrounding system can take. Pillar connects findings to that business context so teams can prioritize the risks in their own environment, and BadEngram is another example of the model-level risks we investigate to inform assessments before production.

The objective is to support a deployment decision: approve an appropriate model, require additional controls, or select an alternative. Organizations can pursue the benefits of open-weight AI while assessing the specific models, components, and connections they intend to trust.

Explore Pillar’s AI Discovery & Posture capabilities, or read the full BadEngram research from Pillar Security and Fujitsu Research of Europe.

‍