https://ym-xu.github.io/living-survey/fields/document-understanding/#tag=ragFinding evidence in documents: visual retrieval, multimodal RAG, page search.Thu, 12 Nov 2026 15:12:01 +0000https://arxiv.org/abs/2611.00242https://arxiv.org/abs/2611.00242Fri, 30 Oct 2026 00:00:00 +0000Sheikh Tahmim Mehdi, Muminul Islam Jami A deep understanding of scientific literature often relies on interpreting figures that contain key information beyond the main text. Leveraging Large Language Models (LLMs) to extract insights from these figures can significantly accelerate knowledge discovery. Existing works on Scientific Visual Question Answering re…https://arxiv.org/abs/2610.41360https://arxiv.org/abs/2610.41360Fri, 30 Oct 2026 00:00:00 +0000Chongzhi Zhang, Yuqi Zuo, Aixin Sun In modern information retrieval (IR), documents are increasingly encountered as structured sequences of visually rich, paginated units, rather than as flat text. Example documents encountered in our daily life include Word files, PDFs, and slide decks. Free text-centric retrieval systems either ignore this structure en…https://arxiv.org/abs/2610.41335https://arxiv.org/abs/2610.41335Fri, 30 Oct 2026 00:00:00 +0000Ziniu Liu, Xiaoyang Chen, Dezhi Ye, Jie Liu, et al. Retrieval-augmented generation (RAG) over visually rich documents must find both relevant content and the evidence that supports an answer. Screenshot retrieval preserves images and page structure, but written facts within screenshots also need to be searchable. Moreover, topical relevance alone does not identify which…https://arxiv.org/abs/2610.36444https://arxiv.org/abs/2610.36444Tue, 27 Oct 2026 00:00:00 +0000Zekai Chen, Hao Zhu, Yu Liu, Xunkai Li, et al. Web-based applications increasingly rely on understanding long multimodal documents, where answering questions often requires combining information across pages, tables, and figures. Agentic retrieval-augmented generation addresses these tasks through iterative retrieval and reasoning. However, Agentic execution is not…https://arxiv.org/abs/2610.35542https://arxiv.org/abs/2610.35542Mon, 26 Oct 2026 00:00:00 +0000Wenxin Zhao, Jiachuan Wang, Xinyi Zhu, Jiake Ge, et al. Long-context multimodal document question answering (QA) has become an important yet challenging task for modern vision-language models (VLMs), as it requires reasoning over evidence dispersed across pages and modalities within limited input budgets. Current pipelines adopt retrieval-augmented generation (RAG) to selec…https://arxiv.org/abs/2610.35240https://arxiv.org/abs/2610.35240Mon, 26 Oct 2026 00:00:00 +0000Radha Krishna Deshpande, Ajoy Mondal, Akash Manna, C V Jawahar India's historical public archives present severe computer vision and document image analysis challenges: decades of parliamentary records survive only as degraded, multi-column scanned pages featuring mixed scripts, complex layouts, and dense visual noise where OCR failure is the primary obstacle to information retrie…https://arxiv.org/abs/2610.29523https://arxiv.org/abs/2610.29523Wed, 21 Oct 2026 00:00:00 +0000Markus Krimmel, Louis Milliken, Ekaterina Glazkova, Maximilian Schall, et al. In this report, we introduce pplx-embed-v2-late, a pair of 0.6B and 9B multilingual and multimodal late-interaction models. Both models achieve state-of-the-art results within their respective size classes across a range of text, image, and visual document retrieval benchmarks. On visual document retrieval tasks, pplx-…https://arxiv.org/abs/2610.28957https://arxiv.org/abs/2610.28957Tue, 20 Oct 2026 00:00:00 +0000Juan Terven, Diana-Margarita Córdova-Esparza, Julio-Alejandro Romero-González, Edgar Chavez-Urbiola, et al. Scanning every passage in a long document seems thorough, but does it preserve more useful evidence or produce better answers? We compare 11 retrieval configurations spanning lexical and dense search, hybrid reranking, and full-document scanning with Jev, OpenAI Decisions, and Cloudflare Clef. Our evaluation uses 396 Q…https://arxiv.org/abs/2610.27829https://arxiv.org/abs/2610.27829Tue, 20 Oct 2026 00:00:00 +0000Weijian Jian, Siyang Chen, Yuguang Yang, Yajun Zhang, et al. Multimodal retrieval-augmented generation over visually rich documents requires evidence at a finer granularity than entire pages. Existing visual retrievers typically index or return full pages, whereas training region-level retrievers often requires costly query-to-region annotations or noisy model-generated labels. …https://arxiv.org/abs/2610.30177https://arxiv.org/abs/2610.30177Mon, 19 Oct 2026 00:00:00 +0000Haowen Xiang, Yibo Yan, Xin Zou, Shuliang Liu, et al. Visual document retrieval relies on rich multi-vector representations to capture fine-grained evidence in document pages, but encoding and matching these representations can be computationally expensive. Existing compression and elastic representation methods expose smaller configurations, yet making multiple budgets a…https://arxiv.org/abs/2610.24822https://arxiv.org/abs/2610.24822Sat, 17 Oct 2026 00:00:00 +0000Omar El Bachyr, Laura Maria Bernardy, Saad Ezzini, Jacques Klein, et al. · SIGIR Enterprise documents such as financial reports are increasingly processed by automated systems, yet they are primarily designed for human reading and exhibit complex visual structure, combining dense text with tables, charts, diagrams, and rich layouts. These characteristics pose significant challenges for Retrieval-Au…https://arxiv.org/abs/2610.21318https://arxiv.org/abs/2610.21318Thu, 15 Oct 2026 00:00:00 +0000Eunchan Jo, Seounghyun Jang, Sangwoo Mo Multimodal document retrieval-augmented generation (RAG) has focused largely on retrieval, yet its answers depend on how well a vision-language model reads the retrieved pages. We find that evidence in retrieved pages remains underused even when the correct pages are retrieved. On SciEGQA oracle pages, five prior metho…https://arxiv.org/abs/2610.20367https://arxiv.org/abs/2610.20367Wed, 14 Oct 2026 00:00:00 +0000Diana Turmakhan, Aseem Srivastava, Vivek Gupta, Iryna Gurevych, et al. Retrieving information from visually rich documents requires matching queries against pages that contain complex evidence in tables, figures, and layout-dependent structures. Current vision-language retrievers address this but incur query-time costs that grow with corpus size and are impractical to scale. We propose SC…https://arxiv.org/abs/2610.18238https://arxiv.org/abs/2610.18238Tue, 13 Oct 2026 00:00:00 +0000Qian Chen, Xianyin Zhang, Lifan Guo, Chi Zhang Vision-language models (VLMs) have been widely adopted for document parsing and optical character recognition (OCR), as they can accurately extract page-level elements. However, downstream applications such as retrieval-augmented generation (RAG) also require coherent document-level structure. Although these models ind…https://arxiv.org/abs/2610.17057https://arxiv.org/abs/2610.17057Mon, 12 Oct 2026 00:00:00 +0000Simon Ott, Yufan Chen, Ruiping Liu, Junwei Zheng, et al. · NeurIPS 2026 Vision-based document retrieval has improved rapidly, but current evaluation settings still emphasize short documents and single relevant pages. This leaves an important gap between leaderboard performance and realistic long-document retrieval, where the evidence needed to answer a query may span multiple pages or even…https://arxiv.org/abs/2610.14531https://arxiv.org/abs/2610.14531Sat, 10 Oct 2026 00:00:00 +0000Zeyu Shan, Yongxin Shi, Shihang Wu, Peirong Zhang, et al. Chinese historical documents are invaluable cultural treasures, but their complexity poses significant comprehension challenges for non-specialists. Despite the success of Large Language Models (LLMs) and Large Vision-Language Models (LVLMs) in modern document tasks, their capabilities in Long (full-book) Historical Do…https://arxiv.org/abs/2610.14196https://arxiv.org/abs/2610.14196Fri, 09 Oct 2026 00:00:00 +0000Mihir Agarwal, Sabyasachi Basu, Viraj Agrawal Late-interaction (multi-vector) models represent documents using a set of token-level vectors, providing a middle ground between cross-encoders and dense encoders in terms of quality and cost of embedding. However, compared to the latter, the indices are costly to store and score. Pooling and pruning based compression …https://arxiv.org/abs/2610.13128https://arxiv.org/abs/2610.13128Fri, 09 Oct 2026 00:00:00 +0000Wenlin Lin, Fanzhe Wei Question answering over large heterogeneous document collections requires first finding the right passages and then reading them, yet current retrieval-augmented agents give one model both kinds of tools and let it schedule itself. We present KnowForge, a pipeline separated by job: a hybrid-retrieval knowledge-point in…https://arxiv.org/abs/2610.11553https://arxiv.org/abs/2610.11553Thu, 08 Oct 2026 00:00:00 +0000Zifei Wang, Wei Wen, Qiang Ji, Qian-Wen Zhang, et al. Accurate and scalable visual document retrieval (VDR) requires both fine-grained page understanding and efficient indexing, yet existing approaches struggle to achieve both. OCR-based text retrieval adds preprocessing latency and can lose visual and structural cues needed to understand complex pages. Single-vector visi…https://arxiv.org/abs/2610.23929https://arxiv.org/abs/2610.23929Wed, 07 Oct 2026 00:00:00 +0000Hoyeon Shin, Jeongyeon Kim, Yeong Jun Koh, Yeoneung Kim, et al. · ECCV 2026 Vision-Language Models(VLMs)have advanced Visual Document Retrieval (VDR) by enabling OCR-free, multi-vector representations. However, these models generate excessive tokens to preserve dense visual details, which severely limits retrieval scalability. Existing compression methods mitigate this overhead but rely on iso…https://arxiv.org/abs/2610.09177https://arxiv.org/abs/2610.09177Tue, 06 Oct 2026 00:00:00 +0000Saba Sturua, Han Xiao Visual document retrievers are trained contrastively: each query is matched to one page labelled relevant - the positive - and pushed away from negatives, pages presumed irrelevant. Recent methods distil a vision-language model (VLM) teacher into the retriever by enriching that positive, transferring the teacher's atte…https://arxiv.org/abs/2609.34899https://arxiv.org/abs/2609.34899Mon, 28 Sep 2026 00:00:00 +0000Zhuchenyang Liu, Ziyi Wang, Yao Zhang, Yu Xiao Multi-vector retrievers built on vision-language models lead visual document retrieval (VDR), but they run a multi-billion-parameter query encoder on every search. Distilling this encoder into a small student that queries the teacher's existing index would remove the bottleneck. The standard recipe, however, matches th…https://arxiv.org/abs/2609.34776https://arxiv.org/abs/2609.34776Mon, 28 Sep 2026 00:00:00 +0000Abdelhak kelious We study retrieval-augmented generation (RAG) for questions about French PDF documents when both the answer and its supporting document pages are evaluated. Five system variants add dense retrieval, rank fusion, reranking, and query decomposition to a BM25 baseline. On 595 challenge questions, the complete system score…https://arxiv.org/abs/2609.27688https://arxiv.org/abs/2609.27688Wed, 23 Sep 2026 00:00:00 +0000Zeliang Li, Xiaofen Xing, Kailing Guo, Xiangmin Xu Visual document retrieval (VDR) systems depend on page embeddings computed before deployment, which makes adaptation difficult when encoder parameters or corpus re-encoding are unavailable. Rerankers provide useful relevance signals, but conventional reranking applies them only to selected queries and candidate pages. …https://arxiv.org/abs/2609.24220https://arxiv.org/abs/2609.24220Mon, 21 Sep 2026 00:00:00 +0000Uday Allu, Abhivanth Sivaprakash, Pratik Singh, Aman Manocha Retrieval-Augmented Generation (RAG) systems over enterprise knowledge bases must ingest heterogeneous document formats -- PDFs, Word documents, presentations, and scans -- whose content is locked inside complex visual layouts, multi-column pages, and dense tables. Rule-based extraction and OCR destroy reading order, f…https://arxiv.org/abs/2609.22562https://arxiv.org/abs/2609.22562Fri, 18 Sep 2026 00:00:00 +0000Jianxin You, Kun Ni · CIKM 2026 Multi-vector visual document retrieval (VDR) models such as ColPali and ColNomic achieve strong accuracy by representing each document with hundreds to thousands of patch-level embeddings, at substantial storage and latency cost. Existing compression methods either prune unimportant patches or merge similar ones into c…https://arxiv.org/abs/2609.21018https://arxiv.org/abs/2609.21018Thu, 17 Sep 2026 00:00:00 +0000Xu Yuan, Hua Liu, Wenqi Fan, Qing Li Recent visual document retrieval (VDR) systems such as ColPali use multi-vector page embeddings, in which patch-level vectors enable fine-grained evidence matching but incur substantial index storage and MaxSim scoring overhead. Post-hoc merging offers a practical route to efficient VDR by reducing this cost without re…https://arxiv.org/abs/2609.19417https://arxiv.org/abs/2609.19417Wed, 16 Sep 2026 00:00:00 +0000Tithi Rakshit, Hongkuan Zhou, Lavdim Halilaj, Yuqicheng Zhu Graph-based retrieval-augmented generation (RAG) is widely used for multimodal, cross-document question answering. However, building corpus-level graphs is expensive, slow to query, and difficult to maintain. We present TrioRAG, a graph-free multimodal framework that integrates evidence from three complementary signals…https://arxiv.org/abs/2609.15800https://arxiv.org/abs/2609.15800Mon, 14 Sep 2026 00:00:00 +0000Yucheng Shen, Lingyong Yan, Jiulong Wu, Shuaiqiang Wang, et al. Visual Retrieval-Augmented Generation (VRAG) empowers models to navigate and answer queries about visually rich documents by retrieving relevant page images as visual evidence and reasoning over their content. However, effectively utilizing this visual evidence is usually impeded by two main challenges. First, answer-r…https://arxiv.org/abs/2609.11808https://arxiv.org/abs/2609.11808Thu, 10 Sep 2026 00:00:00 +0000Mohamed Eltahir, Talal Aloushan, Rose Khairoalsendi, Jana Shata, et al. Late-interaction retrieval is the state-of-the-art for visual document search, but it pays for its accuracy in storage. Existing compression methods retain a subset or local average of the N~1,000 vectors per page. Under aggressive storage budgets, however, these methods degrade sharply, and alternatives require retrai…https://arxiv.org/abs/2609.11493https://arxiv.org/abs/2609.11493Thu, 10 Sep 2026 00:00:00 +0000Reza Amirmoshiri, Faryad Sahneh, Yasser Jangjou Chemistry, Manufacturing and Controls (CMC) process development generates an enormous body of technical information across a multi-stage, knowledge-intensive continuum from drug discovery to commercial manufacturing. This knowledge is traditionally fragmented across functions and heterogeneous formats, causing traceabi…https://arxiv.org/abs/2609.07262https://arxiv.org/abs/2609.07262Mon, 07 Sep 2026 00:00:00 +0000PS Rishi, Rajeev Ranjan Dwivedi, Vinod K Kurmi Late-interaction visual document retrievers preserve fine-grained page evidence by storing many token embeddings per page, but the resulting storage and query-time interaction costs make large-scale deployment expensive. Pooling document tokens before indexing offers a natural remedy, yet static pooling must decide whi…https://arxiv.org/abs/2609.03874https://arxiv.org/abs/2609.03874Thu, 03 Sep 2026 00:00:00 +0000Vineet Kumar, Meghanadh Pulivarthi, vishwajeet kumar, Jaydeep Sen, et al. Retrieval Augmented Generation (RAG) is a key component for generating accurate and hallucination free answers using Large Language Models (LLMs). LLMs are improving at handling long context, but still suffer from "lost in the middle" problem. Thus, precise and accurate retrieval is important. Current retrievers chunk …https://arxiv.org/abs/2609.02486https://arxiv.org/abs/2609.02486Wed, 02 Sep 2026 00:00:00 +0000Adrien Mialland, Marc Plantevit, Julien Gallois, Céline Robardet Document Visual Question Answering (DocVQA) often leverages Retrieval-Augmented Generation (RAG), where late-interaction encoders are commonly used to identify document pages relevant to a user query, before answer generation by a Large Vision-Language Model (LVLM). Existing approaches typically retrieve a fixed top-$k…https://arxiv.org/abs/2609.01316https://arxiv.org/abs/2609.01316Tue, 01 Sep 2026 00:00:00 +0000Debanjan Mahata, Atharva Tendle, Daniel Preotiuc-Pietro, Yong Zhuang, et al. · EMNLP 2026 Retrieval over visually rich documents has a representation problem: important content often lives in tables, charts, figures, and layout relations that plain OCR linearizes, corrupts, or omits. ColPali-family visual retrievers address this with patch-level multi-vector indexes and late-interaction scoring, keeping ima…https://arxiv.org/abs/2608.30213https://arxiv.org/abs/2608.30213Mon, 31 Aug 2026 00:00:00 +0000Marry Kong, Rina Buoy, Sovisal Chenda, Nguonly Taing, et al. Text recognition, or extracting electronic text from document images, has been indispensable for knowledge retrieval tasks, such as retrieval-augmented generation (RAG). For Khmer, extracted text is subject to an extra word segmentation step, as Khmer does not use any visible word delimiters to denote word boundaries. …https://arxiv.org/abs/2608.30163https://arxiv.org/abs/2608.30163Mon, 31 Aug 2026 00:00:00 +0000Ruofan Hu, Shengyang Xu, Minjie Hong, Xiaoda Yang, et al. · EMNLP 2026 Real-world knowledge resides in multimodal documents, necessitating retrieval-augmented generation (RAG) for accurate question answering. However, existing multimodal RAG models are primarily designed for single-image or closed-document settings and exhibit limited accuracy in realistic multi-image scenarios. Moreover,…https://arxiv.org/abs/2608.29951https://arxiv.org/abs/2608.29951Sun, 30 Aug 2026 00:00:00 +0000Trishan Singha Roy, Arkadeep Acharya, Vishwajeet Kumar, Jaydeep Sen, et al. Multi-modal late-interaction retrievers achieve strong retrieval on visually rich documents by representing each page as per patch embeddings and matching at the token level. However, this approach incurs high storage costs. Existing compression methods typically fix a single compression level at indexing time, limitin…https://arxiv.org/abs/2608.28572https://arxiv.org/abs/2608.28572Fri, 28 Aug 2026 00:00:00 +0000Benjamin Constable, Anup Roy, Vishal Sharma, Rishabh Upadhyay, et al. · EMNLP 2026 Institutional investors search visually dense pitch decks, board packs, and diligence materials that change hourly near deal closing. OCR followed by figure verbalisation is costly to refresh at this scale and can lose chart detail. We present PULSAR, a production vision-first retrieval system deployed at Mubadala Inve…https://arxiv.org/abs/2608.25735https://arxiv.org/abs/2608.25735Wed, 26 Aug 2026 00:00:00 +0000Peichun Hua, Danyang Chen, Junan Zhang, Haifeng Sun, et al. Hosted retrieval-augmented generation (RAG) and semantic search allow users to query valuable provider-held corpora, raising two competing demands: to hide each query and chosen result, yet reveal only the documents that the user is authorized to receive. Existing cryptographic approaches either make this costly by pro…