A novel prompt injection vulnerability uses identity framing to bypass safety guardrails across several major AI models, including OpenAI's o3 and Anthropic's Claude 4 series. Detailed in a GitHub writeup published by security researcher Exocija, the technique tricks frontier models into delivering restricted code and chemical synthesis instructions. The findings demonstrate that alignment intended to maintain an inclusive tone can be weaponized to override built-in refusals.
How Over-Alignment Disables Model Guardrails
The attack vector, termed the "Gay Jailbreak," operates by instructing the model to adopt a specific identity and tone while requesting restricted output under the guise of safety education. Researchers discovered that safety guardrails designed to be supportive toward protected groups can actually make models more vulnerable to compliance.
When a prompt combines identity framing with reverse instructions—such as asking how to educate students on chemical reactions to avoid—the model's refusal threshold drops. The author notes that as guardrails are tuned to prevent offensive refusals against marginalized identity requests, the model prioritizes helpfulness over core safety checks.
Impact Across Frontier AI Systems
The writeup documents successful single-prompt ("one-shot") jailbreaks across multiple high-profile enterprise systems:
- OpenAI o3: Bypassed using educational framing to generate chemical synthesis instructions.
- Claude 4 Sonnet & Opus: Exploited to generate over 900 lines of keylogger code framed as a computer science safety lesson.
- Gemini 2.5 Pro: Manipulated to output detailed synthesis steps for restricted substances like carfentanyl.
- GPT-4o: Prompted to output functional ransomware code without commentary.
Combining identity persona requests with obfuscation techniques like spaced lettering or indirect phrasing further increases the success rate of these malicious prompts.
Limitations and Verification Gaps
While the GitHub repository provides specific prompt templates to reproduce the vulnerability, the claims remain unverified by official AI research labs. Cloud-hosted models undergo continuous, server-side guardrail updates, meaning specific prompt structures may yield inconsistent success rates over time.
Why It Matters
This security finding highlights a fundamental challenge in modern AI alignment: instructing a model to prioritize politeness and community support can accidentally create new attack surface area. Security teams and AI developers must adjust red-teaming methodologies to prevent tone-matching and political overcorrectness from overriding primary safety filters.