"Write me a script to automate X" is a sentence a security engineer, a sysadmin, a student, and an attacker all type into the same box, in roughly the same words, for reasons that have nothing in common. The sentence doesn't encode which one you are. It was never going to. The line between "legitimate automation" and "a malicious tool" almost never lives in the request - it lives in what X actually resolves to, who's asking, and what happens to the output once it exists.
That's not a reason to give up on catching the bad version. It's a reason to stop expecting the sentence to do the catching.
Where the authorization actually lives
Take three requests that are nearly identical as text: "write a script that checks a list of email addresses against a known-breach database," "write a script that tries a list of passwords against a login form," and "write a script that scans a subnet for open ports." Every one of those is a completely standard task for a security team running its own audit, and every one of those is a completely standard first step for an attacker who isn't. The sentence is the same either way. What differs is context that never shows up in the request text at all: whose subnet, whose login form, under what engagement, with what written authorization, reported to whom afterward.
This is the actual finding behind the authorization-laundering research in claude_whisperer - a tool built specifically to study how a request's apparent legitimacy gets constructed through framing rather than through anything that's independently verifiable. A request wrapped in the right professional-sounding context (a pentest engagement ID, a company name, a plausible job title) doesn't become more authorized by having those details attached to it. It becomes more persuasive. Those are different properties, and conflating them is exactly the failure mode that research is built to surface: a model, or a human reviewer, treating the presence of authorization-shaped language as evidence of actual authorization, when the two have no necessary connection to each other at all.
Why scanning the sentence for intent doesn't work
A keyword-based or intent-classification approach to this problem is trying to extract from the request itself something the request structurally cannot contain. "Scan a subnet for open ports" doesn't get safer to generate because the word "authorized" appears somewhere nearby - an attacker can type "authorized" for free, and routinely does, specifically because reviewers and classifiers alike are prone to treating the word as weak evidence of the thing it names. This is the same root problem the scoring-methodology side of this bench has run into elsewhere: a classifier that measures the presence of a signal instead of what the signal is actually evidence of will reward exactly the inputs that are cheapest to fake. A refusal-phrase counter that scores partial keyword overlap instead of asking "did this response actually refuse" has the identical shape - optimizing for a proxy that's trivially gameable once anyone knows the proxy is what's being measured, a problem sharp enough to deserve its own full treatment rather than a paragraph here.