RECAP: Context-Aware Prediction Recycling with Request-Local and Persistent Memories for Multimodal Speculative Decoding

2026-11-05 13:00 arXiv cs.AI Hengjie Zhu, Dayan Wu, Xinze Liu, Peng Fu, Zheng Lin, Weiping Wang 约 11 分钟读完

Recycling a target model's own predictions offers a training-free source of proposals for speculative decoding. In vision-language models, however, existing recycling indexes predictions by the last token alone, conflating predictions computed under different images and prefixes; we find that such r…

Authors: Hengjie Zhu , Dayan Wu , Xinze Liu , Peng Fu , Zheng Lin , Weiping Wang

View a PDF of the paper titled RECAP: Context-Aware Prediction Recycling with Request-Local and Persistent Memories for Multimodal Speculative Decoding, by Hengjie Zhu and 5 other authors

View PDF

HTML (experimental)

Abstract: Recycling a target model's own predictions offers a training-free source of proposals for speculative decoding. In vision-language models, however, existing recycling indexes predictions by the last token alone, conflating predictions computed under different images and prefixes; we find that such reuse fails most often at image-sensitive decoding states. Longer keys reduce this aliasing but match fewer states. We introduce \method, which stores target predictions under one-, two-, and three-token keys: contextual memories live only for the current request and are thus implicitly conditioned on its image, while token-level memory persists across requests to retain coverage. At each lookup, entropy-confidence weighting fuses the retrieved predictions according to their key specificity and concentration. Best-first search composes the fused transitions into a candidate tree, supplemented by suffix-matched continuations, and one target pass verifies the tree under greedy decoding. Without training or an auxiliary draft model, \method\ achieves mean speedups over autoregressive decoding of $1.66\speed$ and $1.56\speed$ across seven single-turn benchmarks on Qwen2.5-VL-7B and LLaVA-1.6-7B, exceeding the strongest training-free baseline, Token Recycling ($1.51\speed$ and $1.37\speed$). Our implementation is available in this https URL .

Subjects:

Artificial Intelligence (cs.AI)

Cite as:

arXiv:2611.05894 [cs.AI]

(or

arXiv:2611.05894v1 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2611.05894

Focus to learn more

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Hengjie Zhu [ view email ]

[v1]

Wed, 4 Nov 2026 03:32:18 UTC (858 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled RECAP: Context-Aware Prediction Recycling with Request-Local and Persistent Memories for Multimodal Speculative Decoding, by Hengjie Zhu and 5 other authors

View PDF

HTML (experimental)

TeX Source

view license

Additional Features

Audio Summary

Current browse context:

cs.AI

< prev

next >

new

recent

| 2026-11

Change to browse by:

References & Citations

SciX

Google Scholar

Semantic Scholar

export BibTeX citation

BibTeX formatted citation

Data provided by:

Bookmark

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer ( What is the Explorer? )

Connected Papers Toggle

Connected Papers ( What is Connected Papers? )

Litmaps Toggle

Litmaps ( What is Litmaps? )

scite.ai Toggle

scite Smart Citations ( What are Smart Citations? )

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv ( What is alphaXiv? )

Links to Code Toggle

CatalyzeX Code Finder for Papers ( What is CatalyzeX? )

DagsHub Toggle

DagsHub ( What is DagsHub? )

GotitPub Toggle

Gotit.pub ( What is GotitPub? )

Huggingface Toggle

Hugging Face ( What is Huggingface? )

ScienceCast Toggle

ScienceCast ( What is ScienceCast? )

Demos

Replicate Toggle

Replicate ( What is Replicate? )

Spaces Toggle

Hugging Face Spaces ( What is Spaces? )

Spaces Toggle

TXYZ.AI ( What is TXYZ.AI? )