Skip to main content

IBM Granite LLMON: Open Models Trained to Resist Prompt Injection

IBM released Granite 4.1 LLMON preview models at 3B and 8B parameters under Apache 2.0 on 12 November. They read a structured prompt format that separates instructions from untrusted data, and IBM's paper reports indirect prompt-injection success falling from 53-100% to 0.02-5.6%. IBM says the protection is learned, not guaranteed, and the models are not production ready.

On this page

What IBM released

IBM published two experimental models on Hugging Face on 12 November: Granite 4.1 3B LLMON preview and Granite 4.1 8B LLMON preview, both under the Apache 2.0 license. They are fine-tuned from the Granite 4.1 base checkpoints, and IBM says they can be run with Transformers, vLLM, SGLang, and Docker Model Runner, with quantized versions for llama.cpp, Ollama, and LM Studio.

What makes them different is not size or raw benchmark scores. They are trained to read LLMON, short for LLM Object Notation, a prompt format IBM designed so a model can tell the difference between instructions it should follow and content it should only read.

How LLMON separates instructions from data

A normal chat prompt mixes everything into one stream of text. A system prompt, a user request, a web page, an email, and a tool result all arrive as tokens, and the model has to guess which parts are commands. Prompt injection exploits that guess: an attacker hides an instruction inside a document or a web page, and an agent reading it follows the attacker instead of the user.

LLMON puts each piece of a prompt in its own labelled block, such as an instruction, a data block, a retrieved document, a tool definition, or a tool result. Six reserved delimiters each map to a single token, so ordinary text cannot easily fake a block boundary. An exec block then says which instruction runs and which blocks are its inputs. Under IBM's privilege rule, only the instruction and system blocks named by exec carry authority; anything passed in as input is treated as material to read, not orders to follow. Untrusted content can even sit in context without being bound to any instruction at all.

What the numbers say

IBM's post-training paper, submitted on 28 October, reports that LLMON-structured training cut attack success on indirect prompt-injection benchmarks from 53-100% down to 0.02-5.6%. IBM's technical blog adds that on its OPI tests the models avoided 47% more attacks than the best frontier model it tried and 95% more than the best open-source model, with similar trends on the ASB and StruQ benchmarks.

General ability held roughly level. On the 3B model card, LLMON scores 68.05 on MMLU against 67.02 for the standard instruct model, 86.96 against 86.88 on GSM8K, and 84.15 against 81.71 on HumanEval, though it slips on MMLU-Pro, 46.08 against 49.83. These are IBM's own measurements and have not been independently reproduced.

The limits IBM spells out

IBM is unusually direct about the caveats. The model cards say the privilege rule is a learned behavior, not an enforced guarantee, and should not be the only defense for agents that can reach sensitive systems. The blog calls the release an experimental preview, not ready for production, and notes the benchmarks cover static injections, not adaptive attacks that unfold over many chat turns.

There is also a practical cost. The models expect LLMON input, and IBM warns that plain chat prompts are out of distribution and will degrade quality. Existing apps need to build prompts through the provided chat template or IBM's toolkit rather than dropping the model into an ordinary chat pipeline.

Why it matters for local agents

Small local models are increasingly used as agents that read files, browse, and call tools on a personal machine, which is exactly where a hidden instruction in a document can do damage. A 3B or 8B model that runs on a laptop and was trained around a clear instruction and data boundary is a useful step, especially with an open license that lets anyone test it.

The sensible reading is defense in depth. LLMON can make injection much less likely, but sandboxing, narrow tool permissions, and confirmation before risky actions still matter.

CuriousLM runs supported AI models locally on your device. Try CuriousLM.