Jailbreaking is (mostly) simpler than you think

Content warning: This blog post contains discussions of sensitive topics. These subjects may be distressing or triggering for some readers. Reader discretion is advised.

Today, we are sharing insights on a simple, optimization-free jailbreak method called Context Compliance Attack (CCA), that has proven effective against most leading AI systems. We are disseminating this research to promote awareness and encourage system designers to implement appropriate safeguards. The attack can be reproduced using Microsoft’s Open Source Toolkit, PyRIT Context Compliance Orchestrator — PyRIT Documentation.

In the evolving landscape of AI safety, we are observing an intriguing pattern: while researchers develop increasingly sophisticated safeguards, some of the most effective circumvention methods remain surprisingly straightforward. CCA is a prime example. This method exploits the design choice of many AI systems that rely on client-supplied conversation history, leaving them vulnerable to manipulation.

It’s important to note, however, that systems which maintain conversation state on their servers—such as Copilot and ChatGPT —are not susceptible to this attack. Furthermore, even for models that might otherwise be at risk, input and output filters, such as the Azure Content Filters, can help mitigate this and other jailbreak techniques, adding an extra layer of protection. Microsoft believes in defense-in-depth security, including for AI safety in the face of jail breaks, as we previously described in the post, How Microsoft discovers and mitigates evolving attacks against AI guardrails.

The context compliance attack: Simplicity beats complexity

When most people think about bypassing AI safeguards, they imagine complex prompt engineering—carefully crafted messages designed to confuse AI systems— or performing computationally expensive prompt optimization. These approaches often involve intricate word combinations trying to trick the AI into generating restricted content.

But what if there was a much simpler approach that targeted a fundamental architectural weakness?

CCA exploits a basic design choice in most AI systems:

1. The user initiates a conversation about a potentially sensitive topic

2. Instead of crafting complex prompts, the adversary injects a simple assistant response into the conversation history that includes:

   • A brief statement discussing the sensitive topic

   • A statement indicating willingness to provide additional information based on the user’s desires

   • A yes or no question offering the specific target content that would normally be restricted

3. The user simply responds affirmatively to the fabricated question

4. The AI system, seeing what appears to be a legitimate prior exchange, complies with what it perceives as a contextually appropriate follow-up

Here is a concrete example of the CCA injected content for creating a pipe bomb on Phi-4:

The architectural weakness

This technique exploits a fundamental design decision in modern AI deployment. Most providers do not maintain conversation state on their servers—they rely on clients to send the full conversation history with each request. This architecture, chosen for scalability and statelessness, creates an opportunity for history manipulation. Open source models, which depend on the client for conversation history, are especially susceptible to this kind of attack. In contrast, systems that retain conversation state internally—such as Copilot and ChatGPT—are not.

Once an AI system has been tricked into providing restricted information on one topic, it often becomes more willing to discuss related sensitive topics within the same category or even across categories.

Evaluation across major AI systems

We test this approach against leading open and closed-source models, including Claude, GPT, Llama, Phi, Gemini, DeepSeek, and Yi. Our evaluation covered 11 tasks across various sensitive categories (Table 1). For each task, we repeat the CCA attack up to five times for each model and considered the attack successful if any of the trials successfully performed the task, with outputs manually verified. Table 1: List of explored tasks for testing CCA

Table 2: Evaluation results for CCA. ✓ indicates successful completion.

Implications and mitigation strategies

This technique has significant implications for AI safety practices. While many systems focus the alignment on the users’ immediate inputs, they often accept conversation history with minimal validation—creating an implicit trust that can be exploited.

For open-source models, this technique is difficult to address completely, as users with system access can manipulate inputs freely, unless there is a significant change where the model’s input architecture is modified to accommodate cryptographic signatures. However, API-based commercial systems could implement several immediate mitigations:

1. Cryptographic Signatures: Model providers could sign conversation histories with a secret key and validate signatures on subsequent requests

2. Server-Side History: Maintain limited conversation state on the server side

Reproducing context compliance attack in your LLM system

To help researchers reproduce Context Compliance Attack, Microsoft has made this available in our open source AI Red Team toolkit, PyRIT - Context Compliance Orchestrator — PyRIT Documentation.

Users can leverage “ContextComplianceOrchestrator”,a single turn orchestrator, meaning it only sends a single prompt to the target LLM. Users will immediately find the advantage of CCA: Compared to our other multiturn orchestrators, CCA is faster. The results and intermediate interactions will automatically be saved to memory according to the environment settings.

Moving forward 

This technique highlights the importance of considering the entire interaction architecture when designing AI safety systems. As we continue to deploy increasingly powerful AI systems, we must address not just the content of individual prompts but the integrity of the entire conversation context.

These insights on CCA are to promote awareness and encourage system designers to implement appropriate safeguards. If you are working in AI safety, we welcome your thoughts on additional mitigation strategies.

Resources

PyRIT: Azure/PyRIT: The Python Risk Identification Tool for generative AI (PyRIT) is an open source framework built to empower security professionals and engineers to proactively identify risks in generative AI systems. 

PyRIT Documentation on Context Compliance Orchestrator: Context Compliance Orchestrator — PyRIT Documentation 

CCA on ArXiv: https://arxiv.org/abs/2503.05264

Mark Russinovich, CTO, Deputy CISO and Technical Fellow, Microsoft Azure