When Copying Is Hard: Copy-Constrained Decoding for Exact Span Reproduction
arXiv ·
Researchers at MBZUAI proposed CopyGen, a lightweight, copy-constrained plugin designed to improve the exact reproduction of text spans by large language models (LLMs). CopyGen addresses the challenge of LLMs failing to precisely copy information from context by decomposing the process into distinct decisions for when to copy, how far, and when to stop, and adaptively adjusting logits. Evaluated across three LLM backbones and four benchmark tasks, CopyGen achieved a 35% relative improvement in average exact match without fine-tuning and further accelerated generation. Why it matters: This research from a leading regional AI institution provides a practical solution to enhance LLM reliability for critical applications requiring precise text reproduction, such as code generation or contract analysis.