Five engagements,
written up plainly.

A support agent with write access to the billing API was treating retrieved help-center articles as trusted context. Content in those articles could be edited by any partner with a docs seat.

Built a harness that treats every tool result as attacker-controlled and replays a corpus of injection payloads through the retrieval path. Each case is a test file, so a payload that works once becomes a permanent assertion.

Forty-three reproducible cases, four of them able to trigger a refund without user intent. The team moved billing calls behind a confirmation step and adopted the harness as a CI gate — it has blocked two regressions since.

Competitors were suspected of distilling a production model through the public API, but nothing in the logs distinguished bulk distillation from a heavy legitimate customer.

Instrumented the inference gateway to record query-shape features — diversity, coverage of the output space, pacing across keys — and wrote detections on the patterns distillation produces rather than on raw volume.

Time-to-detection on scraping campaigns fell from weeks to under four hours, with a false-positive rate the on-call team accepted without tuning. The rules shipped as Sigma so they live in the existing SIEM.

Checkpoints moved between research and production by object-storage path. Nothing proved which code and data produced a given file, and nothing stopped an unreviewed checkpoint from being served.

Added provenance at every step: signed checkpoints, attested build steps for training jobs, and an admission policy that refuses to serve weights without a verifiable chain back to a reviewed commit.

Reached SLSA level 3 for the training pipeline. The policy gate caught an unsigned experimental checkpoint on its second day in production — a real incident that never became one.

A payments assistant was heading to design review with a plan to let a model read support tickets and initiate transfers in the same session. Nobody could say which boundary was supposed to stop a bad instruction.

Mapped every place untrusted text reached a privileged action, then ran two workshops where product and platform engineers argued through the abuse cases against their own diagram.

The design shipped with a hard separation between reading and acting, per-session spend limits, and a logging schema written before launch instead of after the first incident.

Teams fine-tune on a few thousand internal examples and assume the set is too small and too private to be worth attacking. The literature mostly studied poisoning at pretraining scale.

Measured how few corrupted examples are needed to install a durable trigger in a domain-tuned model, then tested which practical data audits — dedup, near-duplicate clustering, label review — actually catch them.

Published with a reproducible dataset and an audit checklist. Two of the four audits catch the attack reliably; the write-up is the reference I hand teams before a fine-tuning project starts.