https://ym-xu.github.io/living-survey/fields/document-understanding/#tag=docvqaVision-language models and question answering over document images.Thu, 12 Nov 2026 15:12:01 +0000https://arxiv.org/abs/2611.12957https://arxiv.org/abs/2611.12957Sat, 07 Nov 2026 00:00:00 +0000Bingdian Zhang, Zixuan Li, Long Bai, Xiaolong Jin, et al. VLM-based agents solve complex tasks by iteratively interacting with environments, and their effectiveness depends heavily on how the environment exposes information and provides tools for acting on it. However, constructing an effective environment for complex multimodal document question answering remains challenging…https://arxiv.org/abs/2611.06449https://arxiv.org/abs/2611.06449Wed, 04 Nov 2026 00:00:00 +0000Boyu Niu, Yingjie Xiong, Shurui Li, Linhan Pang, et al. Traditional document parsing systems use Markdown as their output representation, which is effective for linear text but does not fully preserve fine-grained spatial structure, table geometry, visual hierarchy, or rendering semantics. We show that HTML is advantageous for heterogeneous and layout-sensitive documents wh…https://arxiv.org/abs/2610.41360https://arxiv.org/abs/2610.41360Fri, 30 Oct 2026 00:00:00 +0000Chongzhi Zhang, Yuqi Zuo, Aixin Sun In modern information retrieval (IR), documents are increasingly encountered as structured sequences of visually rich, paginated units, rather than as flat text. Example documents encountered in our daily life include Word files, PDFs, and slide decks. Free text-centric retrieval systems either ignore this structure en…https://arxiv.org/abs/2610.36929https://arxiv.org/abs/2610.36929Tue, 27 Oct 2026 00:00:00 +0000Saifullah Saifullah, David Dembinsky, Stefan Agne, Andreas Dengel, et al. Multimodal transformer models have driven unprecedented performance breakthroughs in modern document understanding. However, the inherent black-box nature of these models remains a major obstacle to their safe deployment in high-stakes sectors. Despite significant progress in explainability for vision and language task…https://arxiv.org/abs/2610.36759https://arxiv.org/abs/2610.36759Tue, 27 Oct 2026 00:00:00 +0000Jing Huang, Zhiya Tan, Changtao Miao, Shutao Gong, et al. Multilingual multi-hop document question answering (MMHop-DocQA) requires locating and integrating evidence across complex layouts, document pages, and diverse languages. Existing approaches include single-pass processing and adaptive tool use, but learning to coordinate evidence acquisition, cross-page integration, an…https://arxiv.org/abs/2610.31385https://arxiv.org/abs/2610.31385Thu, 22 Oct 2026 00:00:00 +0000Yufei Tong, Guanjie Cheng, Tianle Wu, Leilei Zhang, et al. Visual document understanding is essential for acquiring knowledge from visually rich documents and is attracting growing research interest. Existing surveys emphasize architectures or technical approaches, with limited attention to how task bottlenecks shape technical choices and research priorities. We present a task…https://arxiv.org/abs/2610.25235https://arxiv.org/abs/2610.25235Sun, 18 Oct 2026 00:00:00 +0000Jumana Alsubhi, Ahmed Alhusayni, Abdulrahman Gharawi, Israa Hamdine, et al. Choosing an OCR system for a multilingual archive requires balancing accuracy against deployment constraints. This study compares cloud and self-hosted systems on three tasks: bounding box detection on 48 Arabic, English, and French pages, text recognition on 315 pages across nine document types, and mathematical expre…https://arxiv.org/abs/2610.24822https://arxiv.org/abs/2610.24822Sat, 17 Oct 2026 00:00:00 +0000Omar El Bachyr, Laura Maria Bernardy, Saad Ezzini, Jacques Klein, et al. · SIGIR Enterprise documents such as financial reports are increasingly processed by automated systems, yet they are primarily designed for human reading and exhibit complex visual structure, combining dense text with tables, charts, diagrams, and rich layouts. These characteristics pose significant challenges for Retrieval-Au…https://arxiv.org/abs/2610.21327https://arxiv.org/abs/2610.21327Thu, 15 Oct 2026 00:00:00 +0000Zhuowen Liang, Xiaotian Lin, Yuyu Luo, Haixun Wang, et al. Multimodal long-document question answering requires models to seek sparse evidence across pages and modalities, understand heterogeneous document content, and reason over the collected evidence. Even advanced vision-language models (VLMs) can overlook sparse evidence and struggle with cross-modal reasoning in a single…https://arxiv.org/abs/2610.17530https://arxiv.org/abs/2610.17530Tue, 13 Oct 2026 00:00:00 +0000Kai Liu, Anqi Li, Junxian Li, Zhixin Wang, et al. We propose a unified spatial-density signal to score token importance for both inference phases: bipartite merge for prefill and KV eviction for decode, yielding measurable gains in each stage. To isolate the two phases, we split benchmarks by scoring logic. Prefill probes (MMBench, POPE, ScienceQA) evaluate only the f…https://arxiv.org/abs/2610.15728https://arxiv.org/abs/2610.15728Sun, 11 Oct 2026 00:00:00 +0000Qingzhe Han, Fan Dang, Shengqi Chen, Xinyi Sun, et al. Visible watermarks convey document ownership, confidentiality, and status, but can also carry adversarial signals. We introduce Auto-Regional Adversarial Watermarking (AR-AWM), a model-aware white-box attack on end-to-end visual language models (VLMs) for document understanding. AR-AWM constructs a small, spatially dis…https://arxiv.org/abs/2610.14531https://arxiv.org/abs/2610.14531Sat, 10 Oct 2026 00:00:00 +0000Zeyu Shan, Yongxin Shi, Shihang Wu, Peirong Zhang, et al. Chinese historical documents are invaluable cultural treasures, but their complexity poses significant comprehension challenges for non-specialists. Despite the success of Large Language Models (LLMs) and Large Vision-Language Models (LVLMs) in modern document tasks, their capabilities in Long (full-book) Historical Do…https://arxiv.org/abs/2609.37712https://arxiv.org/abs/2609.37712Tue, 29 Sep 2026 00:00:00 +0000GuangJian Team, Kaili Huang, Yongshuo Zhang, Bingtao Fu, et al. Optical Character Recognition (OCR) is evolving from plain-text transcription toward general visual intelligence, requiring models to recognize, localize, and reason over textual information in complex visual environments. However, existing OCR systems often excel at only some tasks and struggle to balance recognition,…https://arxiv.org/abs/2609.35304https://arxiv.org/abs/2609.35304Mon, 28 Sep 2026 00:00:00 +0000Antonios Georgakopoulos, Paul Groth, Lise Stork Multimodal knowledge graphs (KGs) integrate information from text, figures, tables, and other modalities into a unified structured representation, with the promise that richer evidence enables better inference. In GraphRAG systems built over such graphs, it is commonly assumed that retrieving evidence from more modalit…https://arxiv.org/abs/2609.32041https://arxiv.org/abs/2609.32041Fri, 25 Sep 2026 00:00:00 +0000Souhail Bakkali, Ayoub Merimi Modern Document AI reads contracts, extracts fields, reasons over tables, and grounds answers to page regions, then forgets everything. Processing an amendment the next day begins from scratch: no schema retained, no contradiction detected, no experience carried forward. This is a structural choice, not a scale failure…https://arxiv.org/abs/2609.24092https://arxiv.org/abs/2609.24092Mon, 21 Sep 2026 00:00:00 +0000Jeremy Cerwin Wang, Wai Kit Wong, Jeff Kai Tai Tang Real-world document processing systems rely on rigid, predefined schemas, yet critical target fields often lack direct visual counterparts on the page. Extracting these implicit values requires multi-hop derivation, such as aggregating sub-categories or reasoning over visual marks. While existing methods handle explici…https://arxiv.org/abs/2609.20574https://arxiv.org/abs/2609.20574Thu, 17 Sep 2026 00:00:00 +0000Luca De Grandis, Silvia Cappelletti, William Raccagni, Marcella Cornia, et al. · BMVC 2026 Answer grounding in document visual question answering remains an open challenge: most benchmarks lack grounding annotations or provide limited-quality labels, while constructing grounded datasets still requires costly manual effort. We introduce DocAttriBench (DAB), a large-scale benchmark for fine-grained, element-le…https://arxiv.org/abs/2609.13815https://arxiv.org/abs/2609.13815Sat, 12 Sep 2026 00:00:00 +0000Ritali Vatsi, Rachapudi Jagadeesh, Shruti Singh Baghel, Himani Sharma, et al. Text-centric Visual Question Answering (VQA) requires reading and reasoning over text embedded in images, a task made substantially harder when images suffer from real-world degradation such as motion blur, low resolution, or compression artifacts. While modular OCR-based pipelines and end-to-end vision-language models…https://arxiv.org/abs/2609.25055https://arxiv.org/abs/2609.25055Tue, 08 Sep 2026 00:00:00 +0000Artemis Llabrés, Marc Serra Ortega, Tomàs Ockier, Samuel Ortega Cuadra, et al. · ICDAR 2026 In this report we present results of the ICDAR2026 Competition on Multimodal Reasoning over Documents in Multiple Domains. This competition aimed to advance research in document understanding through the task of Visual Question Answering (VQA). Building upon previous DocVQA benchmarks, this competition introduces chall…https://arxiv.org/abs/2609.13268https://arxiv.org/abs/2609.13268Mon, 07 Sep 2026 00:00:00 +0000Amirul Rahman, Aisha Karim, Kenji Nakamura, Yi-Fan Ng Annual reports, diligence packs, and infographic dashboards bury numbers in page images: axes, cell grids, and footnotes that OCR pipelines flatten and that page-level visual retrievers still treat as interchangeable in-context examples. We keep a frozen Qwen2.5-VL-7B-Instruct generator and a ColPali / VisRAG-Ret page …https://arxiv.org/abs/2609.05141https://arxiv.org/abs/2609.05141Fri, 04 Sep 2026 00:00:00 +0000Shenxi Wu, Yuhong Liu, Haosong Zhang, Tongjin Zou, et al. Scientific papers require models to integrate evidence across text, equations, figures, tables, code, and datasets while preserving its provenance. Beyond answer correctness, scientific reading requires verifiable outputs from operations such as evidence localization, definition extraction, and consistency checking. We…https://arxiv.org/abs/2609.02486https://arxiv.org/abs/2609.02486Wed, 02 Sep 2026 00:00:00 +0000Adrien Mialland, Marc Plantevit, Julien Gallois, Céline Robardet Document Visual Question Answering (DocVQA) often leverages Retrieval-Augmented Generation (RAG), where late-interaction encoders are commonly used to identify document pages relevant to a user query, before answer generation by a Large Vision-Language Model (LVLM). Existing approaches typically retrieve a fixed top-$k…https://arxiv.org/abs/2609.02207https://arxiv.org/abs/2609.02207Wed, 02 Sep 2026 00:00:00 +0000Vishnu Prasad Vijaya Kumar, Santhosh Venkatesh, Ivan P. Yamshchikov · EMNLP 2026 Real-world personally identifiable information (PII) redaction often operates on document images---scans, screenshots, and PDF renderings---where OCR errors, layout structure, and visual noise determine whether sensitive information is actually removed. Existing PII benchmarks are mostly text-centric and do not measure…https://arxiv.org/abs/2609.01575https://arxiv.org/abs/2609.01575Tue, 01 Sep 2026 00:00:00 +0000Maksim Evdokimov, Matvey Ivanov, Dmitrii Tsiupin, Olga Tsymboi, et al. Extracting structured fields from hundreds of millions of documents annually remains costly in regulated industries: bespoke OCR cascades cover only a fraction of workflows, privacy rules preclude external models, and existing open-source VLMs that clear quality thresholds cost more to serve than human annotation. We p…https://arxiv.org/abs/2609.01316https://arxiv.org/abs/2609.01316Tue, 01 Sep 2026 00:00:00 +0000Debanjan Mahata, Atharva Tendle, Daniel Preotiuc-Pietro, Yong Zhuang, et al. · EMNLP 2026 Retrieval over visually rich documents has a representation problem: important content often lives in tables, charts, figures, and layout relations that plain OCR linearizes, corrupts, or omits. ColPali-family visual retrievers address this with patch-level multi-vector indexes and late-interaction scoring, keeping ima…https://arxiv.org/abs/2609.00756https://arxiv.org/abs/2609.00756Tue, 01 Sep 2026 00:00:00 +0000Chengguang Gan, Yunhao Liang, Hanjun Wei, Qinghao Zhang, et al. The Mutual Reinforcement Effect (MRE) asks whether a fine, span-level and a coarse, document-level task help each other when one model handles both. We test it in multimodal document understanding on three corpora, two of receipts and one of scanned business forms, comparing single-task, joint and conditioned training,…https://arxiv.org/abs/2608.30678https://arxiv.org/abs/2608.30678Mon, 31 Aug 2026 00:00:00 +0000Gengxu Li, Yuan Wu, Yi Chang · EMNLP 2026 Findings Text-rich image understanding requires multimodal large language models (MLLMs) to organize OCR (Optical Character Recognition)-grounded evidence across words, layout, fields, charts, and visual correspondences. Existing evaluations often conflate extraction with reasoning and rarely test whether models follow the requ…https://arxiv.org/abs/2608.28784https://arxiv.org/abs/2608.28784Fri, 28 Aug 2026 00:00:00 +0000Jinlong Li, Jiaming Ding, Dingfu Lu, Malcolm Hsiu, et al. · ECCV 2026 Multimodal Large Language Models (MLLMs) have recently made strong progress in visual--linguistic understanding. However, their performance on text-centric video reasoning remains highly sensitive to input quality. Real-world user-provided videos often contain motion blur, compression artifacts, noise, and low-resoluti…https://arxiv.org/abs/2608.26949https://arxiv.org/abs/2608.26949Thu, 27 Aug 2026 00:00:00 +0000Iñigo Alonso, Mirella Lapata Answering questions over real-world documents requires processing long inputs that interleave text with tables. Optical context compression, which represents context as images, promises to reduce token cost, but its effect on table understanding remains unclear. We study pixel-level table compression for question answe…https://arxiv.org/abs/2608.25618https://arxiv.org/abs/2608.25618Wed, 26 Aug 2026 00:00:00 +0000Dongzhuoran Zhou, Yuqicheng Zhu, Yule Liu, Zhen Yang, et al. · EMNLP 2026 Findings Long-document visual question answering increasingly relies on VLM agents that retrieve candidate pages, inspect page images, write findings to working memory, and synthesize answers. Working memory should carry answer-supporting evidence across page inspections for later grounded answering, yet existing evaluation mai…https://arxiv.org/abs/2608.24011https://arxiv.org/abs/2608.24011Tue, 25 Aug 2026 00:00:00 +0000Yuchuan Wu, Xuan Luo, Yinglian Zhu, Meng Fang, et al. Chinese ancient document understanding demands complex visual, linguistic, and historical reasoning. Current Large Vision-Language Models (LVLMs) typically rely on an opaque, single-pass generation paradigm, often producing overconfident and weakly grounded responses. To address this, we propose SAGE, an evidence-groun…https://arxiv.org/abs/2608.21853https://arxiv.org/abs/2608.21853Sat, 22 Aug 2026 00:00:00 +0000Sławomir Dadas, Michał Perełkiewicz, Rafał Poświata, Małgorzata Grębowiec, et al. Large language models are increasingly moving beyond text processing, adding support for other modalities such as images and audio. While text understanding and generation have been extensively studied, multimodal data processing capabilities, particularly in the context of cultures and languages other than English, ha…https://arxiv.org/abs/2609.22124https://arxiv.org/abs/2609.22124Thu, 20 Aug 2026 00:00:00 +0000Illya Havrylov This paper describes the system submitted to the UNLP 2026 Shared Task on Multi-Domain Document Understanding. The challenge required extracting precise answers, document IDs, and page numbers from a diverse corpus of Ukrainian PDF documents within a strict 9-hour offline Kaggle execution limit. During evaluation on th…https://arxiv.org/abs/2608.19739https://arxiv.org/abs/2608.19739Thu, 20 Aug 2026 00:00:00 +0000Alin-Ionut Popa Multimodal LLMs can see a document, but they often can't read it reliably. Small text, tables, visual cues, and topological elements still trip them up under direct visual inference, even when the page is already sitting in the model's context. Most document-VQA systems treat perception as fixed: they encode the page o…https://arxiv.org/abs/2608.17889https://arxiv.org/abs/2608.17889Tue, 18 Aug 2026 00:00:00 +0000Lexiang Hu, Yanzhao Zhang, Mingxin Li, Dingkun Long, et al. Visually rich documents encode relevance through language, layout, structured visual elements, and corpus context, yet retrieval is typically evaluated by one-shot query--page matching. Agentic-search benchmarks usually score downstream question answering or report generation, leaving document ranking under iterative e…https://arxiv.org/abs/2608.16417https://arxiv.org/abs/2608.16417Mon, 17 Aug 2026 00:00:00 +0000Hao Zhang, Longrong Yang, Lunhao Duan, Ziyang Wang, et al. Multi-modal retrieval-augmented generation (RAG) is a key technique for visually rich long document understanding. Existing multi-modal RAG methods are progressively advancing toward multi-agent systems: they first retrieve relevant pages based on a query, and then iteratively understand information within those pages.…https://arxiv.org/abs/2608.12911https://arxiv.org/abs/2608.12911Thu, 13 Aug 2026 00:00:00 +0000Beining Xu, Hairui Wang, Jiaxin Wang, Changsheng Chen, et al. · ACM MM 2026 While the privacy risks of multimodal large language models (MLLMs) have drawn significant attention, the unique vulnerabilities of domain-specific MLLMs remain largely underexplored. Focusing on document understanding MLLMs for identity document processing, this paper investigates the privacy issues inherent in Key In…https://arxiv.org/abs/2608.10628https://arxiv.org/abs/2608.10628Tue, 11 Aug 2026 00:00:00 +0000Kaican Li, Weiyan Xie, Lewei Yao, Jiannan Wu, et al. · EMNLP 2026 Long-document understanding often requires reasoning over many visually rich pages, making inference costly and prone to context rot. In this work, we propose InSight-doc, an agentic visual perception framework that treats visual resolution as an adaptive reasoning-time resource. InSight-doc starts from low resolution …https://arxiv.org/abs/2608.08512https://arxiv.org/abs/2608.08512Sun, 09 Aug 2026 00:00:00 +0000Mahbub E Sobhani, Md. Faiyaz Abdullah Sayeedi, Fahmid Hasan Chowdhury, Md Adnan Arefeen, et al. Evolving documents, such as laws, tax codes, and software documentation, are amended, replaced, and sometimes reverted over time, so a question has different correct answers at different dates. In contrast to encyclopedic knowledge, where an old fact is simply overwritten, an amendment is itself an official text that s…https://arxiv.org/abs/2608.28635https://arxiv.org/abs/2608.28635Sat, 08 Aug 2026 00:00:00 +0000Nimol Thuon, Panhapin Theang · ICPR 2026 Recent multimodal large language models (MLLMs) have advanced document understanding, visual question answering, and text extraction. However, their reliability in low-resource, non-Latin settings remains uncertain. Khmer form documents present particular challenges because they contain complex script forms, mixed Khme…