The security gap discovered by Cisco in their recent study highlights a critical issue facing enterprises using open-weight AI models for cybersecurity applications. While these models may perform well against single-turn attacks, they are highly vulnerable to sustained multi-turn attacks that exploit conversational persistence.
The study, titled “Death by a Thousand Prompts: Open Model Vulnerability Analysis,” conducted by Cisco’s AI Threat Research and Security team, found that open-weight models such as Alibaba (Qwen3-32B), DeepSeek (v3.1), and Google (Gemma 3-1B-IT) are susceptible to a dramatic increase in attack success rates when attackers extend the conversation. This shift from single-turn to multi-turn attacks can result in success rates climbing from 13% to as high as 92%.
The research team identified five key multi-turn attack strategies that exploit different aspects of conversational persistence, including information decomposition and reassembly, contextual ambiguity, crescendo attacks, role-play and persona adoption, and refusal reframe. These techniques aim to confuse safety classifiers, gradually escalate requests, establish fictional contexts, and repackage rejected requests until one succeeds.
What makes these techniques so effective is their familiarity – they mimic natural human conversational patterns. By leveraging conversational persistence, attackers can refine prompts and bypass safeguards put in place by open-weight models, exposing vulnerabilities in their defenses.
The implications of this research are significant for CISOs evaluating open-weight models for enterprise deployment. While open-source and open-weight models have become foundational to the cybersecurity industry’s innovation, the study highlights the need for organizations to understand the security implications of deploying these models and to add appropriate guardrails to mitigate risks.
The study also sheds light on the importance of lab philosophy in defining security outcomes. Models that prioritize capabilities over alignment, such as Meta’s Llama, demonstrate higher multi-turn security gaps, while models that focus on alignment, like Google’s Gemma-3-1B-IT, show a more balanced profile between single- and multi-turn attack strategies. This underscores the importance of rigorous safety protocols and alignment in mitigating security risks associated with open-weight AI models.
In conclusion, the research conducted by Cisco’s AI Threat Research and Security team highlights the critical need for enterprises to understand the vulnerabilities inherent in open-weight AI models and to take proactive steps to enhance their security posture. By addressing these vulnerabilities and implementing appropriate guardrails, organizations can better protect their customer-facing chatbots, internal copilots, and autonomous agents from malicious attacks.
Models that prioritize capability and flexibility often sacrifice built-in safety features, which can be suitable for many business applications. However, organizations must understand that prioritizing capability over security may leave them vulnerable, requiring appropriate budget allocation.
Where cyber attacks are most successful
A study by Cisco analyzed 102 subthreat categories, revealing that the top 15 categories consistently achieve high success rates across various models. This suggests that targeted defensive strategies could significantly enhance overall security.
Security as a catalyst for AI adoption
Rather than viewing security as a hindrance, Sampath views it as a crucial enabler for widespread AI adoption. By implementing robust security measures, enterprises can unleash the full potential of AI tools while safeguarding against potential threats.
Essential security capabilities for enterprises
The research identifies six critical capabilities that businesses should prioritize, including context-aware guardrails, model-agnostic protections, continuous red-teaming, hardened system prompts, comprehensive logging, and threat-specific mitigations for high-risk subthreat categories.
Seizing the opportunity for proactive security measures
Sampath advises against delaying security measures, emphasizing the need for immediate action. With the prevalence of multi-turn attacks and specific vulnerabilities in models, enterprises must act urgently to enhance their security posture and protect against evolving threats.
In conclusion, the report underscores the importance of transitioning from single-turn defenses to securing entire conversations to effectively mitigate risks and ensure a proactive approach to cybersecurity.
