$M^3$: A Multi-LLM System for Multi-Turn Jailbreaking as a Markov Decision Process

2026-10-26 12:00 arXiv cs.AI Yuqing Huang, Jack Ji, Jithin Pradeep, Sagarika Suresh, Jekaterina Novikova 约 11 分钟读完

Modern Large Language Models (LLMs) remain vulnerable to carefully orchestrated multi-turn interactions that progressively bypass safety mechanisms. However, many existing red-teaming approaches rely on heuristics or impose strong assumptions that limit their applicability in fully black box setting…

Authors: Yuqing Huang , Jack Ji , Jithin Pradeep , Sagarika Suresh , Jekaterina Novikova

View a PDF of the paper titled $M^3$: A Multi-LLM System for Multi-Turn Jailbreaking as a Markov Decision Process, by Yuqing Huang and 4 other authors

View PDF

HTML (experimental)

Abstract: Modern Large Language Models (LLMs) remain vulnerable to carefully orchestrated multi-turn interactions that progressively bypass safety mechanisms. However, many existing red-teaming approaches rely on heuristics or impose strong assumptions that limit their applicability in fully black box settings. We address this gap by introducing $M^3$, an automated red-teaming framework that formalizes multi-turn jailbreaking as a finite-horizon Markov Decision Process (MDP). In $M^3$, an attacker LLM interacts with a target LLM while operating over a compact internal state produced by a state LLM. Across two benchmarks and six target models, $M^3$ consistently outperforms state-of-the-art multi-turn red-teaming methods in Attack Success Rates (ASR), achieving improvements of up to 53% in ASR@5. Even with lightweight attacker, $M^3$ attains gains up to 26% in ASR@1 and 34% in ASR@5, compared to baselines that use higher capacity attackers. Beyond improved effectiveness, $M^3$ exhibits steeper risk escalation over conversational turns and achieves more consistent performance across risk categories (up to 4.82 improvement in Gini index) without relying on predefined risk taxonomy, providing a scalable and principled framework for multi-turn red-teaming and safety evaluation.

Subjects:

Artificial Intelligence (cs.AI)

Cite as:

arXiv:2610.31937 [cs.AI]

(or

arXiv:2610.31937v1 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2610.31937

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Yuqing Huang [ view email ]

[v1]

Thu, 22 Oct 2026 21:35:21 UTC (10,595 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled $M^3$: A Multi-LLM System for Multi-Turn Jailbreaking as a Markov Decision Process, by Yuqing Huang and 4 other authors

View PDF

HTML (experimental)

TeX Source

view license

Additional Features

Audio Summary

Current browse context:

cs.AI

< prev

next >

new

recent

| 2026-10

Change to browse by:

References & Citations

SciX

Google Scholar

Semantic Scholar

export BibTeX citation

BibTeX formatted citation

Data provided by:

Bookmark

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer ( What is the Explorer? )

Connected Papers Toggle

Connected Papers ( What is Connected Papers? )

Litmaps Toggle

Litmaps ( What is Litmaps? )

scite.ai Toggle

scite Smart Citations ( What are Smart Citations? )

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv ( What is alphaXiv? )

Links to Code Toggle

CatalyzeX Code Finder for Papers ( What is CatalyzeX? )

DagsHub Toggle

DagsHub ( What is DagsHub? )

GotitPub Toggle

Gotit.pub ( What is GotitPub? )

Huggingface Toggle

Hugging Face ( What is Huggingface? )

ScienceCast Toggle

ScienceCast ( What is ScienceCast? )

Demos

Replicate Toggle

Replicate ( What is Replicate? )

Spaces Toggle

Hugging Face Spaces ( What is Spaces? )

Spaces Toggle

TXYZ.AI ( What is TXYZ.AI? )