https://ym-xu.github.io/living-survey/fields/rag/#tag=multimodalRetrieving images, video, tables and document pages, not only text.Thu, 12 Nov 2026 15:12:01 +0000https://arxiv.org/abs/2611.13675https://arxiv.org/abs/2611.13675Tue, 10 Nov 2026 00:00:00 +0000Senhao Liu, Yuheng Zhang, Chunyu Wei, Yueguo Chen, et al. · NeurIPS 2026 Retrieval-augmented generation (RAG) is bottlenecked by the token budget of large language models: feeding long documents into the context window is expensive, while aggressive textual compression discards fine-grained information that is irreversibly lost in a one-dimensional token sequence. We argue that this bottlen…https://arxiv.org/abs/2611.11960https://arxiv.org/abs/2611.11960Mon, 09 Nov 2026 00:00:00 +0000Guoqing Zhang, Rafik Hadfi, Takayuki Ito Unplanned 30-day intensive care unit (ICU) readmission is a critical benchmark for evaluating healthcare quality. Most predictive models assume readmission risk depends solely on a patient's individual pathophysiological state, ignoring hospital operational context and resource capacity. In practice, discharge decision…https://arxiv.org/abs/2611.10657https://arxiv.org/abs/2611.10657Sat, 07 Nov 2026 00:00:00 +0000Haoran Luo, Ziyue Zhu, Shangyang Wu, Linhao Luo, et al. · NeurIPS 2026 Driven by increasingly complex real-world applications, Retrieval-Augmented Generation (RAG) has evolved from textual pipelines to multimodal settings, comprising two complementary dimensions: knowledge organization and retrieval enhancement. However, existing methods still face challenges of high construction cost, co…https://arxiv.org/abs/2611.08935https://arxiv.org/abs/2611.08935Fri, 06 Nov 2026 00:00:00 +0000Jing Hao, Weinan Zhou, Zhihong Lu, Yaling Shen, et al. Reliable dermatology AI requires access to verifiable evidence distributed across heterogeneous textual and visual sources. However, conventional retrieval-augmented generation (RAG) typically relies on one-shot retrieval from flat corpora, limiting its ability to adapt retrieval granularity, explore multi-hop relation…https://arxiv.org/abs/2611.01409https://arxiv.org/abs/2611.01409Sun, 01 Nov 2026 00:00:00 +0000Elyas Irankhah, Shahed Salehzehi, Parisa Hosseini Purpose: Retrieval-augmented generation (RAG) can introduce electrocardiogram (ECG) criteria that lack patient-specific measurement support. This study evaluates the reliability of retrieved diagnostic criteria in AI-generated ECG reports. Methods: We analyzed 1,083 human-validated PTB-XL patient records using a librar…https://arxiv.org/abs/2610.41335https://arxiv.org/abs/2610.41335Fri, 30 Oct 2026 00:00:00 +0000Ziniu Liu, Xiaoyang Chen, Dezhi Ye, Jie Liu, et al. Retrieval-augmented generation (RAG) over visually rich documents must find both relevant content and the evidence that supports an answer. Screenshot retrieval preserves images and page structure, but written facts within screenshots also need to be searchable. Moreover, topical relevance alone does not identify which…https://arxiv.org/abs/2610.40971https://arxiv.org/abs/2610.40971Thu, 29 Oct 2026 00:00:00 +0000Jiacheng Ding, Yanliang Qi, Cong Guo, Peihua Mai We present FieldGuideRAG, a training-free GraphRAG method in which a vision-language model, or VLM, recognizes fine-grained species by comparing the query with real photographs stored on the nodes of a knowledge graph. Multimodal GraphRAG systems describe each entity in text, and text conveys little of the visual detai…https://arxiv.org/abs/2610.37080https://arxiv.org/abs/2610.37080Tue, 27 Oct 2026 00:00:00 +0000Peichun Hua, He Sun, Zirui Zhuang, Qi Qi, et al. Web retrieval-augmented generation (RAG) splits a page into fixed-size text chunks or, in recent visual pipelines, screenshot tiles, and the retrieved item serves at once as the unit of indexing, ranking, and reading. We trace answer evidence from source HTML to reader input over the full English Wikipedia and find tha…https://arxiv.org/abs/2610.36650https://arxiv.org/abs/2610.36650Tue, 27 Oct 2026 00:00:00 +0000Amiri Hayes, Kristina Wicke Graph-structured representations of text data have become foundational across machine learning, yet it remains unclear how to systematically construct a graph from a given dataset for a target task. Choices about which entities become nodes, which relationships become edges, and what text each node contains can substan…https://arxiv.org/abs/2610.36444https://arxiv.org/abs/2610.36444Tue, 27 Oct 2026 00:00:00 +0000Zekai Chen, Hao Zhu, Yu Liu, Xunkai Li, et al. Web-based applications increasingly rely on understanding long multimodal documents, where answering questions often requires combining information across pages, tables, and figures. Agentic retrieval-augmented generation addresses these tasks through iterative retrieval and reasoning. However, Agentic execution is not…https://arxiv.org/abs/2610.35542https://arxiv.org/abs/2610.35542Mon, 26 Oct 2026 00:00:00 +0000Wenxin Zhao, Jiachuan Wang, Xinyi Zhu, Jiake Ge, et al. Long-context multimodal document question answering (QA) has become an important yet challenging task for modern vision-language models (VLMs), as it requires reasoning over evidence dispersed across pages and modalities within limited input budgets. Current pipelines adopt retrieval-augmented generation (RAG) to selec…https://arxiv.org/abs/2610.35240https://arxiv.org/abs/2610.35240Mon, 26 Oct 2026 00:00:00 +0000Radha Krishna Deshpande, Ajoy Mondal, Akash Manna, C V Jawahar India's historical public archives present severe computer vision and document image analysis challenges: decades of parliamentary records survive only as degraded, multi-column scanned pages featuring mixed scripts, complex layouts, and dense visual noise where OCR failure is the primary obstacle to information retrie…https://arxiv.org/abs/2610.34503https://arxiv.org/abs/2610.34503Sun, 25 Oct 2026 00:00:00 +0000Haoming Huang, Shinpei Hayashi, Kazuya Kohata, Myeongki Kim, et al. Requirements traceability links support activities such as change-impact analysis, coverage checking, and validation. Industrial requirements can include traceable images, which text-only approaches cannot process directly. For a fixed classifier configuration, binary decisions alone offer no direct way to adjust the b…https://arxiv.org/abs/2610.33277https://arxiv.org/abs/2610.33277Fri, 23 Oct 2026 00:00:00 +0000Abdul Hannan Azad, Irvin Wang, Jiwan Jiang, Chenchen Kuai, et al. Engineering standards are challenging knowledge sources for large language models (LLMs) because they are hierarchical, multimodal, and densely connected through figures, tables, cross-references, and normative rules. Existing retrieval-augmented generation (RAG) methods often rely on unstructured text retrieval and st…https://arxiv.org/abs/2610.31880https://arxiv.org/abs/2610.31880Thu, 22 Oct 2026 00:00:00 +0000Meng Liang, Fucheng Wang, Lang Zhao, Jun Chen, et al. The Yangtze River Basin is a critical region for freshwater biodiversity conservation. The implementation of the Yangtze River Ten-Year Fishing Ban has increased the importance of effective aquatic biodiversity monitoring and species-level identification. However, fine-grained identification of closely related freshwat…https://arxiv.org/abs/2610.25575https://arxiv.org/abs/2610.25575Sun, 18 Oct 2026 00:00:00 +0000Qiwei Han, Chi Zhou, Gaoxiang Yuan While retrieval-augmented language models increasingly serve as literature assistants in the physical sciences, existing benchmarks evaluate them primarily on qualitative text overlap or through model-based scoring rather than against verified experimental ground truth. In experimental materials disciplines such as per…https://arxiv.org/abs/2610.24822https://arxiv.org/abs/2610.24822Sat, 17 Oct 2026 00:00:00 +0000Omar El Bachyr, Laura Maria Bernardy, Saad Ezzini, Jacques Klein, et al. · SIGIR Enterprise documents such as financial reports are increasingly processed by automated systems, yet they are primarily designed for human reading and exhibit complex visual structure, combining dense text with tables, charts, diagrams, and rich layouts. These characteristics pose significant challenges for Retrieval-Au…https://arxiv.org/abs/2610.23588https://arxiv.org/abs/2610.23588Fri, 16 Oct 2026 00:00:00 +0000Daniela Jimenez-Pano, Hugo Jair Escalante, Delia Irazú Hernández-Farías Medical Vision-Language Models (Med-LVLMs) have shown strong performance in Medical Visual Question Answering (MVQA), yet they may still produce factually inconsistent responses. Retrieval-Augmented Generation (RAG) can mitigate this issue by incorporating external clinical evidence; however, retrieval is often guided …https://arxiv.org/abs/2610.21318https://arxiv.org/abs/2610.21318Thu, 15 Oct 2026 00:00:00 +0000Eunchan Jo, Seounghyun Jang, Sangwoo Mo Multimodal document retrieval-augmented generation (RAG) has focused largely on retrieval, yet its answers depend on how well a vision-language model reads the retrieved pages. We find that evidence in retrieved pages remains underused even when the correct pages are retrieved. On SciEGQA oracle pages, five prior metho…https://arxiv.org/abs/2610.18238https://arxiv.org/abs/2610.18238Tue, 13 Oct 2026 00:00:00 +0000Qian Chen, Xianyin Zhang, Lifan Guo, Chi Zhang Vision-language models (VLMs) have been widely adopted for document parsing and optical character recognition (OCR), as they can accurately extract page-level elements. However, downstream applications such as retrieval-augmented generation (RAG) also require coherent document-level structure. Although these models ind…https://arxiv.org/abs/2610.15463https://arxiv.org/abs/2610.15463Sun, 11 Oct 2026 00:00:00 +0000Hexiang Gu, Saihui Hou, Changlan Liu, Min Ren, et al. With the rapid proliferation of social media, harmful memes have become a growing threat to online ecosystems. Their harmfulness is often implicitly conveyed through complex interactions among visual, textual, and contextual cues, making accurate detection challenging. Recent studies have introduced Retrieval-Augmented…https://arxiv.org/abs/2610.14531https://arxiv.org/abs/2610.14531Sat, 10 Oct 2026 00:00:00 +0000Zeyu Shan, Yongxin Shi, Shihang Wu, Peirong Zhang, et al. Chinese historical documents are invaluable cultural treasures, but their complexity poses significant comprehension challenges for non-specialists. Despite the success of Large Language Models (LLMs) and Large Vision-Language Models (LVLMs) in modern document tasks, their capabilities in Long (full-book) Historical Do…https://arxiv.org/abs/2610.14428https://arxiv.org/abs/2610.14428Sat, 10 Oct 2026 00:00:00 +0000Istok Bako, Bonan Min, Tiffany Shao, Claire Cheng, et al. · EMNLP 2026 Enterprise workflow automation typically requires retrieving a complete and accurate set of relevant information from large operational document corpora (SOPs, policy documents) given a specific scenario or request. In customer support (CS), for example, human specialists must determine the correct sequence of resoluti…https://arxiv.org/abs/2610.13128https://arxiv.org/abs/2610.13128Fri, 09 Oct 2026 00:00:00 +0000Wenlin Lin, Fanzhe Wei Question answering over large heterogeneous document collections requires first finding the right passages and then reading them, yet current retrieval-augmented agents give one model both kinds of tools and let it schedule itself. We present KnowForge, a pipeline separated by job: a hybrid-retrieval knowledge-point in…https://arxiv.org/abs/2610.12085https://arxiv.org/abs/2610.12085Thu, 08 Oct 2026 00:00:00 +0000Ali Satvaty, Narjes Sharafi, Jirui Qi, Suzan Verberne, et al. · EMNLP 2026 Large language models (LLMs) can expose memorized training sequences under prefix-based extraction: given a prefix from a training example, the model may assign high probability to the original continuation. In deployed systems, however, prefixes are rarely evaluated in isolation. They often appear together with instru…https://arxiv.org/abs/2610.11228https://arxiv.org/abs/2610.11228Thu, 08 Oct 2026 00:00:00 +0000Jason Marcell Setiadi, Xin Cao, Lina Yao Multimodal Large Language Models (MLLMs) have demonstrated strong potential for sequential recommendation through their ability to reason over complex multimodal data. However, existing approaches either rely solely on the target user's own interaction history, neglecting collaborative signals from neighboring users, o…https://arxiv.org/abs/2610.09360https://arxiv.org/abs/2610.09360Wed, 07 Oct 2026 00:00:00 +0000Ruochi Li, Jianzhe Lin, Haoxuan Zhang, Haihua Chen, et al. · NeurIPS 2026 Real-world documents distribute evidence across text, tables, figures, and captions within complex page layouts. Answering complex questions over such documents therefore requires more than retrieving relevant passages: systems must recover the evidence topology that connects heterogeneous evidence units. Existing Grap…https://arxiv.org/abs/2610.07622https://arxiv.org/abs/2610.07622Tue, 06 Oct 2026 00:00:00 +0000Prince Larbi Ampofo, Ryoji Kubo, Djellel Difallah Recent advancements in large language models have introduced new capabilities for reasoning over structured data, particularly through program-aided tools that can analyze tables. However, many existing methods address single-table scenarios or assume that the relevant tables are already provided. In practice, users of…https://arxiv.org/abs/2610.03421https://arxiv.org/abs/2610.03421Fri, 02 Oct 2026 00:00:00 +0000Hang Gao, Wujiang Xu, Zhixing Zhang, Kai Mei, et al. · EMNLP 2026 Findings Multimodal large language models (MLLMs) have shown strong visual reasoning abilities, but knowledge-intensive visual question answering often requires external textual evidence beyond the image and the model's parametric knowledge. Existing multimodal RAG systems commonly rely on Top-$K$ retrieval or reranking, which …https://arxiv.org/abs/2610.01871https://arxiv.org/abs/2610.01871Thu, 01 Oct 2026 00:00:00 +0000Maria Carmen Jica, Ali Satvaty, Suzan Verberne, Fatih Turkmen Multimodal Retrieval-Augmented Generation (MRAG) has emerged as a reliable and cost-effective technique of grounding the generative capabilities of Multimodal Large Language Models (MLLMs) into relevant, up-to-date, external knowledge. Despite presenting several benefits, such as reducing hallucinatory behavior, they a…https://arxiv.org/abs/2610.00923https://arxiv.org/abs/2610.00923Thu, 01 Oct 2026 00:00:00 +0000Hyojeong Yun, Jueun Kim, Wook-Shin Han Multimodal RAG retrieves text, tables, images, and videos, but choosing a retrieval granularity does not determine how much context to retain within each item. Coarse units include irrelevant content, while uniformly fine selection can remove context needed to interpret the evidence. Existing compressors address this t…https://arxiv.org/abs/2610.20632https://arxiv.org/abs/2610.20632Wed, 30 Sep 2026 00:00:00 +0000Rajdeep Pandhare, Rohit Kundaikar, Yakub Kojar, Shreyansh Tiwari Modern healthcare generates increasingly complex medical reports that challenge clinical professionals in terms of time and interpretive burden. Manual review processes are slow, error-prone, and difficult to scale. This paper presents MeddeX, a Medical Report Analyzer that combines a QLoRA fine-tuned LLaMA-3 (8B) mode…https://arxiv.org/abs/2609.37426https://arxiv.org/abs/2609.37426Tue, 29 Sep 2026 00:00:00 +0000Arka Mukherjee, Kaleen Shrestha, Larissa Zhu, Maja Matarić Modern vision-language models (VLMs) have shown promising results in long-video understanding due to the rich semantic information they can capture. However, most methods focus on coarse captioning of extracted image frames that are computationally inefficient and require models with large context windows. While past w…https://arxiv.org/abs/2609.37349https://arxiv.org/abs/2609.37349Tue, 29 Sep 2026 00:00:00 +0000Yalun Wu, Bingzhou Wang, Boyang Wang, Peiying Wang, et al. Multi-step visual retrieval-augmented generation (RAG) answers complex questions by repeatedly retrieving visual evidence, updating an intermediate state, and deciding whether to continue searching or answer. Yet retrieving relevant evidence does not ensure its effective use throughout the reasoning trajectory. As mult…https://arxiv.org/abs/2609.36326https://arxiv.org/abs/2609.36326Mon, 28 Sep 2026 00:00:00 +0000Truong Son Nguyen, Daniel Blackley, Ni Trieu, Evgenios M. Kornaropoulos Retrieval-augmented generation (RAG) hands the user's query to whoever hosts the corpus. We propose PILLAR, a Privacy-Preserving RAG (PPRAG) system based on Private Information Retrieval (PIR) in which a client utilizes the k documents most similar to their query from a server-held and publicly known corpus to respond …https://arxiv.org/abs/2609.35304https://arxiv.org/abs/2609.35304Mon, 28 Sep 2026 00:00:00 +0000Antonios Georgakopoulos, Paul Groth, Lise Stork Multimodal knowledge graphs (KGs) integrate information from text, figures, tables, and other modalities into a unified structured representation, with the promise that richer evidence enables better inference. In GraphRAG systems built over such graphs, it is commonly assumed that retrieving evidence from more modalit…https://arxiv.org/abs/2609.35086https://arxiv.org/abs/2609.35086Mon, 28 Sep 2026 00:00:00 +0000Kelvin J. L. Koa, Xinyang Li, Ke-Wei Huang · NeurIPS 2026 In this work, we study portfolio optimization under the stochastic discount factor (SDF) framework by learning market state representations that capture the underlying risk structures of financial data. This is challenging due to several factors: financial markets exhibit non-stationary dynamics with shifting regimes, …https://arxiv.org/abs/2609.34179https://arxiv.org/abs/2609.34179Mon, 28 Sep 2026 00:00:00 +0000Richard Krueger, Lucas Krause, Zach Pocquette Retrieval-augmented generation systems are extensively instrumented with metrics, benchmarks, traces, and automated judges, but these tools do not decide whether a proposed policy change is safe to release. We present RAGWarrant, an open-source promotion-control framework that treats deployment as a constrained evidenc…https://arxiv.org/abs/2609.31397https://arxiv.org/abs/2609.31397Fri, 25 Sep 2026 00:00:00 +0000Andrea Masini, Sudipta Acharya, Paolo Bellavista, Luca Foschini, et al. Automated and highly usable Quality-of-Service (QoS) enforcement requires translating high-level service intents into deployable traffic-management policies. Although intent-based networking (IBN) has simplified policy specification, bridging the gap between business-level intents and executable network configurations …https://arxiv.org/abs/2609.35810https://arxiv.org/abs/2609.35810Sun, 20 Sep 2026 00:00:00 +0000Jizheng Lai, Yingyun Li, Ying Qin, Haiyang Qian · EMNLP 2026 Large language models are increasingly used in oncology applications, but their predictions are often weakly grounded in explicit medical structure. We present TRACE, a deployable tree-relational enhancement framework for oncology LLMs. TRACE separates expensive offline structure learning from lightweight online infere…