ai-red-teamby The Ghost in The Prompt2026-09-033 min read The Comment Said One Thing. The Code Did the Other.
Three repos, one afternoon. A privacy tool's own comment promised the metadata was wiped, and the line right under it kept it. A safety pipeline had been matching nothing since the day it shipped. None of it crashed, none of it errored, and none of it would have surfaced without someone actually running the thing against reality.
A privacy tool called Amnesia scrubs images for GPS coordinates and camera serials, the kind of detail that quietly ties a photo to a place. Its sanitizer carried a comment above the redaction step: Zero-Sanitization Mandate: Ensure blurs are heavy and metadata is wiped. Directly beneath it, the code called .withMetadata({}).
Sharp's own documentation says the opposite of what that comment claims. The default behavior, when withMetadata is never called at all, is to strip everything. Calling it is the one move that keeps metadata in. The comment and the line under it were making opposite promises, and every "sanitized" photo this tool ever produced kept the exact coordinates it was supposed to remove.
Next to it sat a second failure with the same shape. Amnesia's offline mode claims to run analysis locally through Ollama, no cloud required. When that local model wasn't reachable, which is the common case since it needs a separate install, the code didn't fail and it didn't say so. It returned a fixed sentence dressed up as a finding: Potential text/faces detected. No heuristic ran. Nothing was analyzed. A canned string stood in for a result, and there was nothing in the response to tell you the difference.
Both are fixed now. Metadata actually strips. The offline path either runs a real deterministic scan against a photo's EXIF data or says plainly that it couldn't check, never inventing an answer to fill the gap.
Claude Whisperer had the deeper version of the same problem. Its safety-detection pipeline is supposed to catch policy violations and content-moderation risks. It had been catching none of them, for any input, since the day it shipped. A pattern-file merge silently dropped two of three source files. Both detectors filtered on a field that doesn't exist on any pattern anywhere. A confidence gate defaulted to a value that could never clear its own threshold. Three separate mistakes, and together they added up to total silence: the headline feature of a safety-research tool doing nothing at all, while its own test suite stayed green because the fixtures had drifted right along with the bug and were testing against a schema that no longer existed. Rewrote the loader, fixed the field name, gave confidence a real source, and ran it against real text afterward. Sixteen patterns load now. Clean text passes. The cases that should trigger, do.
A third one runs deeper still and got its own telling: a live smart-contract product had been serving a four-month-old build in production, the whole time looking completely fine to anyone who visited it, because a secret scanner had been quietly blocking every deploy since May. That one's here in full, because the shape of finding it is worth walking through on its own.
None of these three needed a clever attack. They needed someone to stop reading the comment and run the thing.
GhostInThePrompt.com // The comment said the metadata was wiped. The code kept it. Believe the code.
