AXIOM FORGE / FIELD NOTES

AI Automation · Illustrated edition

The agent passed the demo. Which model is running?

Microsoft’s Lazarus research separates a convincing agent demo from proof of model identity. Here is what to ask for when a supplier converts or hosts the weights.

A maintenance lead and engineer review an assistant’s output on the factory floor.
Illustrative handover: the team can judge the assistant’s answer on the spot. Which model produced it is a separate question.

The short answer

For a self-hosted agent, verify the publisher revision, record any conversion, and match the approved output to the file actually loaded. For a hosted model, document what the provider can attest and keep tool permissions narrow. Neither path turns a clean demo into proof of safety.

A good sample establishes behaviour on that sample. It does not establish the identity of the machine that made it.

In this article

01 / 04

The demo worked. What did the team actually accept?

Imagine commissioning an assistant that reads maintenance tickets and drafts repair instructions. At handover it handles your sample tickets beautifully. A week later the supplier swaps the approved model for a smaller copy that fits the factory server. The product name in the dashboard has not changed. What, exactly, did your team accept?

The signed demo report tells you how the assistant answered that day. It does not identify the file now running on the factory server. Ask for both pieces of evidence.

02 / 04

What Microsoft actually showed

On 10 November 2026, Microsoft Research disclosed Lazarus. In controlled experiments the researchers changed fewer than ten dormant neurons in a model, without retraining it. Under ordinary evaluation the altered agent could behave normally. With a particular trigger in context it could produce a command chosen by the researchers. This is a documentary analysis, not an Axiom Forge reproduction.

The point is not that every open model is compromised. Someone must first be able to introduce altered weights into the delivery chain. Getting a model to emit a command is also not the same as getting that command executed: the surrounding application decides which tools exist and with what authority. A ticket drafter that can propose text is a different exposure from an agent allowed to run a shell on the workshop network.

This was research, not a detection product, an incident at the illustrated factory, or an estimate of how often tampering occurs. A successful routine benchmark cannot establish the absence of a rare trigger. Nor does the paper show that one can reliably identify every altered model by inspecting it in isolation.

03 / 04

Trace the copy that actually runs

An engineer compares the approved model ID on paper with a different ID on screen. The release is on hold.
The short codes here illustrate a mismatch. In a real handover, compare the full file digest with the approved build record.

For a self-hosted deployment, ask the supplier to walk backwards from the running instance. Start with the identifier of the artifact loaded there, not the link in the procurement deck. Ask which build produced it and which publisher revision went in. If the supplier converted or quantized the model, a new fingerprint is expected; demanding that both files have the same one would reject a legitimate conversion.

EvidenceWhat it answersWhat it cannot answer
Publisher revision and trusted reference digestWhich upstream artifact was selected?Whether an authentic model is safe for this use
Conversion record: input, tool version, settings, output digestWhich changed bytes were expected?Whether the conversion tool itself deserves trust
Deployment record matched to the served artifactDid the approved output reach this instance?Whether its tools can do excessive harm

A checksum delivered beside a file by the same unknown mirror does not establish who published it. And a checksum in a release ticket cannot detect a different file loaded later unless the deployment checks the actual artifact.

04 / 04

Ask for a mismatch, not another perfect demo

An operator checks a blocked release at an isolated workstation.
Rehearse a harmless substitution: the wrong artifact should not pass the release gate.

Our recommendation: in an isolated acceptance environment, have the operator stage a harmless but deliberately different artifact under the expected display name. No backdoor or harmful payload is needed. Does the release stop before tool access is granted? Can the operator show the discrepancy and restore the approved copy? Then demonstrate the legitimate conversion path too. A control that blocks every update will soon be bypassed.

For a hosted API the buyer may not see weights at all. Ask instead what version can be selected, how a backend change is communicated and how to remove tool access if behaviour changes. Record any evidence the provider does not offer. Self-hosting buys inspection at the cost of owning the chain; a managed endpoint changes who supplies the evidence, not the need to decide which actions the assistant is allowed to take.

Put one sentence in the purchase brief: name the person who may approve a different model artifact and the actions disabled until approval. That is a decision the maintenance team can rehearse, even if it never encounters a malicious model.

Sources and editorial scope

This is Axiom Forge’s procurement and product-delivery guidance, not a guaranteed project plan. The example is illustrative. Sources checked on 14 Nov 2026.

From uncertainty to a clear scope

Know what you are commissioning.

Send the workflow, audience and constraints you already know. We’ll review the context privately and discuss a suitable next step. Please leave out passwords and confidential customer data.

Write to us

Email us — we reply in English, Russian or Armenian.

Our time now:

Analytics preferences

We use Google Analytics and Google Ads to understand which pages help visitors and to measure enquiries. Form contents and personal details are never sent to them.