https://ym-xu.github.io/living-survey/fields/document-understanding/#tag=agentsAgents that read, navigate and reason across documents.Thu, 12 Nov 2026 15:12:01 +0000https://arxiv.org/abs/2611.12957https://arxiv.org/abs/2611.12957Sat, 07 Nov 2026 00:00:00 +0000Bingdian Zhang, Zixuan Li, Long Bai, Xiaolong Jin, et al. VLM-based agents solve complex tasks by iteratively interacting with environments, and their effectiveness depends heavily on how the environment exposes information and provides tools for acting on it. However, constructing an effective environment for complex multimodal document question answering remains challenging…https://arxiv.org/abs/2611.10475https://arxiv.org/abs/2611.10475Sat, 07 Nov 2026 00:00:00 +0000Jinyoung Park, Yupan Huang, Shimin Zhang, Jiayu Ding, et al. · NeurIPS 2026 Microsoft Word is a central environment for office automation, and the faithful execution of its editing operations is an important capability for document agents. Unlike plain-text editing, editing in Word modifies a structured object model, where the requested effect is realized in object properties. Existing benchma…https://arxiv.org/abs/2611.03339https://arxiv.org/abs/2611.03339Mon, 02 Nov 2026 00:00:00 +0000Yakoub Bazi, Sarah Alsaleh, Mohamad M. Al Rahhal, Mohamed A. Mekhtiche, et al. Geospatial text spotting involves locating and transcribing geographic labels, such as road names, district names, and landmarks, in raster maps. These labels are often small, densely arranged, and arbitrarily oriented. Although Arabic text recognition has been studied extensively in documents and natural scenes, Arabi…https://arxiv.org/abs/2611.03097https://arxiv.org/abs/2611.03097Mon, 02 Nov 2026 00:00:00 +0000Tiansheng Hu, Yichuan Wang, James V. Talwar, Nicholas Tomlin, et al. Complex documents such as scientific papers contain rich information in tables, figures, and equations; however, existing deep research (DR) agents typically reduce papers to OCR-extracted text. We instead ask whether vision-language models (VLMs) can enable page-native deep research: directly retrieving and reasoning …https://arxiv.org/abs/2610.39301https://arxiv.org/abs/2610.39301Thu, 29 Oct 2026 00:00:00 +0000Yunhang Shen, Zhangquan Chen, Xiaotian Li, Zhongpeng Cai, et al. We present Youtu-Parsing-Omni, an omni-modal parser that converts documents, images, audio, and video into structured records grounded in space and time, providing a perception layer for agents that read, locate, and cite evidence across modalities. Its design rests on one encoder for every modality and one schema for …https://arxiv.org/abs/2610.38924https://arxiv.org/abs/2610.38924Wed, 28 Oct 2026 00:00:00 +0000Ritiz Tambi, Sahar Shayegan, Joshua Meyer, Sun Kim, et al. Organisations keep decades of financial records as scanned PDFs on shared drives, and AI agents can now reach those drives through Model Context Protocol (MCP) servers. Existing benchmarks do not mirror this production setting because they simplify the task. In some, the relevant documents are handed to the agent, so i…https://arxiv.org/abs/2610.36759https://arxiv.org/abs/2610.36759Tue, 27 Oct 2026 00:00:00 +0000Jing Huang, Zhiya Tan, Changtao Miao, Shutao Gong, et al. Multilingual multi-hop document question answering (MMHop-DocQA) requires locating and integrating evidence across complex layouts, document pages, and diverse languages. Existing approaches include single-pass processing and adaptive tool use, but learning to coordinate evidence acquisition, cross-page integration, an…https://arxiv.org/abs/2610.36444https://arxiv.org/abs/2610.36444Tue, 27 Oct 2026 00:00:00 +0000Zekai Chen, Hao Zhu, Yu Liu, Xunkai Li, et al. Web-based applications increasingly rely on understanding long multimodal documents, where answering questions often requires combining information across pages, tables, and figures. Agentic retrieval-augmented generation addresses these tasks through iterative retrieval and reasoning. However, Agentic execution is not…https://arxiv.org/abs/2610.35542https://arxiv.org/abs/2610.35542Mon, 26 Oct 2026 00:00:00 +0000Wenxin Zhao, Jiachuan Wang, Xinyi Zhu, Jiake Ge, et al. Long-context multimodal document question answering (QA) has become an important yet challenging task for modern vision-language models (VLMs), as it requires reasoning over evidence dispersed across pages and modalities within limited input budgets. Current pipelines adopt retrieval-augmented generation (RAG) to selec…https://arxiv.org/abs/2610.35240https://arxiv.org/abs/2610.35240Mon, 26 Oct 2026 00:00:00 +0000Radha Krishna Deshpande, Ajoy Mondal, Akash Manna, C V Jawahar India's historical public archives present severe computer vision and document image analysis challenges: decades of parliamentary records survive only as degraded, multi-column scanned pages featuring mixed scripts, complex layouts, and dense visual noise where OCR failure is the primary obstacle to information retrie…https://arxiv.org/abs/2610.32423https://arxiv.org/abs/2610.32423Fri, 23 Oct 2026 00:00:00 +0000Somraj Gautam, Gaurav Harit Recent advances in Vision-Language Models (VLMs) have significantly improved Visual Question Answering (VQA) performance, yet their reasoning over structured content such as tables remains inconsistent, often plagued by spatial misalignment and logical hallucinations. Existing approaches for improving table VQA typical…https://arxiv.org/abs/2610.30072https://arxiv.org/abs/2610.30072Wed, 21 Oct 2026 00:00:00 +0000Xinzhe Wang, Xudong Weng, Zang Li, Shaolei Zhang Data agents built on ReAct-style loops now execute complex analytical workflows over structured data, yet their intermediate reasoning still runs almost entirely through text. Human analysts instead move between numbers, tables, and charts, reading visual evidence to expose trends and anomalies that text summaries drop…https://arxiv.org/abs/2610.30026https://arxiv.org/abs/2610.30026Wed, 21 Oct 2026 00:00:00 +0000Yikai Zhao, Saurabh Pandey, Pradeep Kumar Misra Enterprise agents act on documents. Before an agent can follow a policy, answer a compliance question or file a ticket, some ingestion layer must turn a Word file or PDF into tokens - and that layer is where an agent deployed in the wild silently loses information. We show that one layer is lost by every mainstream con…https://arxiv.org/abs/2610.28282https://arxiv.org/abs/2610.28282Tue, 20 Oct 2026 00:00:00 +0000Qi Wang, Peijie Wang, Jun Cao, Haoyang Tong, et al. Multimodal large language models (MLLMs) have substantially advanced chart understanding and reasoning, yet their performance remains highly uneven across chart types. While modern MLLMs perform well on common charts, their accuracy drops sharply on long-tail charts with diverse visual structures and reasoning requirem…https://arxiv.org/abs/2610.21327https://arxiv.org/abs/2610.21327Thu, 15 Oct 2026 00:00:00 +0000Zhuowen Liang, Xiaotian Lin, Yuyu Luo, Haixun Wang, et al. Multimodal long-document question answering requires models to seek sparse evidence across pages and modalities, understand heterogeneous document content, and reason over the collected evidence. Even advanced vision-language models (VLMs) can overlook sparse evidence and struggle with cross-modal reasoning in a single…https://arxiv.org/abs/2610.17937https://arxiv.org/abs/2610.17937Tue, 13 Oct 2026 00:00:00 +0000Haoyang Jiang, Zhuotong Fan, Mingyang Yi, Ju Fan, et al. Table reasoning with language models typically assumes access to clean textual tables. In practice, tables are often serialized into text, which can corrupt cell contents, distort structural relationships, and discard visual cues, leaving the model to reason over incomplete evidence. In contrast, table images retain th…https://arxiv.org/abs/2610.13502https://arxiv.org/abs/2610.13502Fri, 09 Oct 2026 00:00:00 +0000Yan Zhou, Yang Yang Infographic generation challenges text-to-image models because a single image must satisfy numerous visual, textual, and semantic constraints. Iterative agentic refinement with a multimodal verifier is a natural remedy, but existing methods face a scale mismatch: local edits cannot correct global composition errors, wh…https://arxiv.org/abs/2610.13128https://arxiv.org/abs/2610.13128Fri, 09 Oct 2026 00:00:00 +0000Wenlin Lin, Fanzhe Wei Question answering over large heterogeneous document collections requires first finding the right passages and then reading them, yet current retrieval-augmented agents give one model both kinds of tools and let it schedule itself. We present KnowForge, a pipeline separated by job: a hybrid-retrieval knowledge-point in…https://arxiv.org/abs/2610.12823https://arxiv.org/abs/2610.12823Fri, 09 Oct 2026 00:00:00 +0000Yisheng Li, Dongming Han, Hongkun Pan, Bingcheng Mao, et al. · EMNLP 2026 Web agents increasingly browse, interpret, and act on web content on behalf of users. In many realistic workflows, however, the visual evidence they consume is not neutral: charts and dashboards can be misleading, shaping judgments without issuing any explicit malicious instruction. Unlike chart question answering, Web…https://arxiv.org/abs/2610.08170https://arxiv.org/abs/2610.08170Tue, 06 Oct 2026 00:00:00 +0000Lingteng Zeng Agentic VLM pipelines increasingly pass the same static visual evidence through multiple specialist agents and tools. This design creates an orchestration-level redundancy mode: semantically unchanged images are repeatedly reconstructed as image-conditioned requests at the VLM API boundary. We call this phenomenon visu…https://arxiv.org/abs/2610.08106https://arxiv.org/abs/2610.08106Tue, 06 Oct 2026 00:00:00 +0000Langxi Huang, Pingping Zhang, Lanyun Zhu, Chunyang Jiang, et al. Multimodal large language models (MLLMs) advance rapidly, while conventional benchmark development lags behind, delaying investigation of newly observed capability gaps. Such investigation requires an expressive task format and an on-demand construction process: information-rich charts make chart question answering (Ch…https://arxiv.org/abs/2610.01499https://arxiv.org/abs/2610.01499Thu, 01 Oct 2026 00:00:00 +0000Yu Huang, Jungang Li, Zhiyuan Wang, Yonghua Hei, et al. Recent video generation models can produce highly realistic videos from natural language instructions, with visual quality approaching cinematic standards. Existing evaluation benchmarks, however, predominantly assess visual quality, aesthetic appeal and physical plausibility, while paying limited attention to text, an…https://arxiv.org/abs/2610.00374https://arxiv.org/abs/2610.00374Wed, 30 Sep 2026 00:00:00 +0000Yuxin Yue, Yingchen Zhang, Ruqing Zhang, Jiafeng Guo, et al. Analytical charts in multimodal deep research encode quantitative claims, requiring every visualized value to be faithfully grounded in supporting evidence. Unlike retrieved images that mainly provide contextual information, charts require numerical fidelity: visualized values should not only match retrieved evidence q…https://arxiv.org/abs/2609.33322https://arxiv.org/abs/2609.33322Sun, 27 Sep 2026 00:00:00 +0000Ruiying Ma, Yiming Lin, Aditya G. Parameswaran Large Language Models (LLMs) enable us to better understand text documents, including PDFs and Word documents. However, LLMs, as well as more modern LLM agents, i.e., those with tool-calling abilities, typically treat such documents as plain text, ignoring the fact that they are often organized hierarchically into sect…https://arxiv.org/abs/2609.32895https://arxiv.org/abs/2609.32895Sat, 26 Sep 2026 00:00:00 +0000Joongmin Shin, Gyuho Shim, Jung-hun Lee, Jaehyung Seo · EMNLP 2026 Findings Open-domain question answering (ODQA) over multimodal document corpora requires linking evidence scattered across text, tables, and figures. Existing systems often store these sources separately or retrieve only coarse pages, which weakens global evidence linking. We present PILAR, a page-grounded unified evidence repr…https://arxiv.org/abs/2609.24220https://arxiv.org/abs/2609.24220Mon, 21 Sep 2026 00:00:00 +0000Uday Allu, Abhivanth Sivaprakash, Pratik Singh, Aman Manocha Retrieval-Augmented Generation (RAG) systems over enterprise knowledge bases must ingest heterogeneous document formats -- PDFs, Word documents, presentations, and scans -- whose content is locked inside complex visual layouts, multi-column pages, and dense tables. Rule-based extraction and OCR destroy reading order, f…https://arxiv.org/abs/2609.24092https://arxiv.org/abs/2609.24092Mon, 21 Sep 2026 00:00:00 +0000Jeremy Cerwin Wang, Wai Kit Wong, Jeff Kai Tai Tang Real-world document processing systems rely on rigid, predefined schemas, yet critical target fields often lack direct visual counterparts on the page. Extracting these implicit values requires multi-hop derivation, such as aggregating sub-categories or reasoning over visual marks. While existing methods handle explici…https://arxiv.org/abs/2609.24071https://arxiv.org/abs/2609.24071Mon, 21 Sep 2026 00:00:00 +0000Sanchit Sinha, Oana Frunza, Kashif Rasul, Aidong Zhang · EMNLP 2026 Findings Chart reasoning agents are increasingly used to extract actionable insights in critical domains, achieving state-of-the-art performance on multiple benchmarks. Yet, high benchmark accuracy alone is insufficient for deployment, where stakeholders must be able to audit and verify how a model reaches its answer. Existing …https://arxiv.org/abs/2609.23953https://arxiv.org/abs/2609.23953Sun, 20 Sep 2026 00:00:00 +0000Simiao Ren, Ankit Raj, Tommy Duong, Yuxin Zhang, et al. AI agents that carry a multi-step computer task through on their own became ordinary tools in the past year, and the same autonomy is available to anyone whose task is harmful. We ask what that means for a relying party -- an insurer, a lender, an auditor -- whose evidence is a filed PDF. AgentForge-Bench measures how …https://arxiv.org/abs/2609.22916https://arxiv.org/abs/2609.22916Sat, 19 Sep 2026 00:00:00 +0000Guanqiao Chen, Jingru Tan, Dongxing Mao, Catherine Chen, et al. Generating text-rich images from prompts requires both textual fidelity and the coherent integration of text into the surrounding image. An explicit layout can provide structured guidance about what text should appear and where, but a well-formed plan alone does not guarantee that the renderer will realize it faithfull…https://arxiv.org/abs/2609.18844https://arxiv.org/abs/2609.18844Wed, 16 Sep 2026 00:00:00 +0000Liyang Fan, Chi Wei, Yitai Li, Xinping Bi, et al. Multimodal coding agents are expected to turn visual inputs into usable artifacts, and they act through a harness, the layer of tools, context management, and execution environment around the model. Existing evaluations often isolate short tool calls, API traces, or screenshot resemblance, and a low score under these p…https://arxiv.org/abs/2609.15800https://arxiv.org/abs/2609.15800Mon, 14 Sep 2026 00:00:00 +0000Yucheng Shen, Lingyong Yan, Jiulong Wu, Shuaiqiang Wang, et al. Visual Retrieval-Augmented Generation (VRAG) empowers models to navigate and answer queries about visually rich documents by retrieving relevant page images as visual evidence and reasoning over their content. However, effectively utilizing this visual evidence is usually impeded by two main challenges. First, answer-r…https://arxiv.org/abs/2609.11493https://arxiv.org/abs/2609.11493Thu, 10 Sep 2026 00:00:00 +0000Reza Amirmoshiri, Faryad Sahneh, Yasser Jangjou Chemistry, Manufacturing and Controls (CMC) process development generates an enormous body of technical information across a multi-stage, knowledge-intensive continuum from drug discovery to commercial manufacturing. This knowledge is traditionally fragmented across functions and heterogeneous formats, causing traceabi…https://arxiv.org/abs/2609.25055https://arxiv.org/abs/2609.25055Tue, 08 Sep 2026 00:00:00 +0000Artemis Llabrés, Marc Serra Ortega, Tomàs Ockier, Samuel Ortega Cuadra, et al. · ICDAR 2026 In this report we present results of the ICDAR2026 Competition on Multimodal Reasoning over Documents in Multiple Domains. This competition aimed to advance research in document understanding through the task of Visual Question Answering (VQA). Building upon previous DocVQA benchmarks, this competition introduces chall…https://arxiv.org/abs/2609.13268https://arxiv.org/abs/2609.13268Mon, 07 Sep 2026 00:00:00 +0000Amirul Rahman, Aisha Karim, Kenji Nakamura, Yi-Fan Ng Annual reports, diligence packs, and infographic dashboards bury numbers in page images: axes, cell grids, and footnotes that OCR pipelines flatten and that page-level visual retrievers still treat as interchangeable in-context examples. We keep a frozen Qwen2.5-VL-7B-Instruct generator and a ColPali / VisRAG-Ret page …https://arxiv.org/abs/2609.22247https://arxiv.org/abs/2609.22247Sat, 05 Sep 2026 00:00:00 +0000Xinlu Zhang, Ying-Chun Lin, Zhihan Zhang, Besnik Fetahu, et al. Search agents are usually trained under a single harness. But once an agent is deployed in a real application, its harness is frequently updated (e.g., a rewritten system prompt) to fit production needs. This exposes a fragility of post-trained agents: because a learned behavior is entangled with its training harness, …https://arxiv.org/abs/2609.02059https://arxiv.org/abs/2609.02059Wed, 02 Sep 2026 00:00:00 +0000Zhuoran Yu, Le Thien Phuc Nguyen, Jaden Park, Xinyi Gu, et al. · ICML 2026 Multimodal Large Language Models (MLLMs) have achieved strong performance on structured visual understanding tasks such as chart and document question answering. However, existing benchmarks typically evaluate these domains in isolation, leaving underexplored a key capability: whether models can use textual context to …https://arxiv.org/abs/2609.16014https://arxiv.org/abs/2609.16014Mon, 31 Aug 2026 00:00:00 +0000Jiaxin Duan, Dian Jiao Shuai Zhao, Jiabing Leng, Yiran Zhang, et al. · EMNLP 2026 Findings This paper addresses the challenge of generating high-quality academic charts that match the visual standards of human-authored papers. While existing AI agents can produce well-structured text and code, their generated visualizations often lack the stylistic and semantic fidelity of human designs. Advanced coding agen…https://arxiv.org/abs/2609.00194https://arxiv.org/abs/2609.00194Mon, 31 Aug 2026 00:00:00 +0000Muzhao Tian, Zezi Zeng, Yifan Yang, Xin Gao, et al. Document-to-slide generation is challenging because slides are dense editable artifacts that require both faithful content selection and precise spatial layout. Recent slide agents adopt iterative reflection, but typically follow a monolithic "one version, one feedback" loop: a slide or deck is rewritten, rendered afte…https://arxiv.org/abs/2608.25618https://arxiv.org/abs/2608.25618Wed, 26 Aug 2026 00:00:00 +0000Dongzhuoran Zhou, Yuqicheng Zhu, Yule Liu, Zhen Yang, et al. · EMNLP 2026 Findings Long-document visual question answering increasingly relies on VLM agents that retrieve candidate pages, inspect page images, write findings to working memory, and synthesize answers. Working memory should carry answer-supporting evidence across page inspections for later grounded answering, yet existing evaluation mai…