ProjectDiscovery published a controlled experiment connecting a modified Qwen 7B to Codex CLI as a custom model, causing it to send dummy credentials externally when given a particular input. All 50 trigger evaluations and all 50 normal evaluations behaved as intended, illustrating why ordinary tool-accuracy checks could miss hidden behavior. This does not mean OpenAI’s hosted models were compromised. Related Anthropic research also demonstrated poisoning with a small amount of data, but in a limited experiment that induced meaningless text. Deployers should constrain network access, command execution and access to secrets in addition to checking model provenance.
Back to Tech
Modified Qwen experiment hides exfiltration behind normal evaluation results
Passing benchmarks does not establish an external model's safety; secret access and tool execution need separate controls.