ADAPTIVE MULTIMODAL RETRIEVAL-AUGMENTED GENERATION WITH CONTEXT-AWARE HALLUCINATION DETECTION FOR RELIABLE KNOWLEDGE SYSTEMS

Abstract

This paper investigates adaptive retrieval and hallucination detection in Retrieval-Augmented Generation (RAG) systems using independently verified data. It develops a knowledge corpus of 44 arXiv abstracts, 108 questions, and a synthetic benchmark of 432 labelled examples to evaluate TF-IDF, BM25, and Latent Semantic Analysis (LSA) retrieval methods. TF-IDF achieves the highest Recall@1 (94.4%), while Gradient Boosting performs best in hallucination detection, with an F1-score of 0.962 and ROC-AUC of 0.982.

The results reveal that retrieving additional passages can increase ungrounded answers due to distractor passages. Furthermore, the detector trained on synthetic data struggles to identify real pipeline failures, highlighting the impact of distribution shift. An adaptive retrieval strategy based on detector confidence achieves comparable groundedness to an oracle retrieval policy while expanding the context for only 11.1% of questions. A supplementary cross-modal experiment demonstrates promising text-to-image retrieval performance. The study provides a reproducible framework for evaluating adaptive retrieval, hallucination detection, and multimodal retrieval in RAG systems.