Skip to content
GCC AI Research

HalluTruthQA: A Fine-Grained Benchmark for Hallucination Detection, Localization, and Explanation in Arabic Question Answering

arXiv · · Significant research

NLP LLM Research Arabic AI

Summary

Researchers introduce HalluTruthQA, a new fine-grained benchmark designed for hallucination detection, localization, and explanation in Arabic question answering. This benchmark comprises 2,400 expert-curated examples across Islamic knowledge, history, science, and geography, featuring character-level error spans, human explanations, and various hallucination types. The study evaluated four open-source Arabic LLMs (ALLaM-7B, Falcon-H1R-7B, Qwen3-32B, SILMA) across detection, localization, factual verification, and explanation tasks, revealing no single model outperforms others across all metrics. Why it matters: HalluTruthQA provides a critical tool for advancing the factual accuracy and reliability of Arabic LLMs by enabling more granular and comprehensive hallucination evaluation beyond response-level detection.

Keywords

Hallucination detection · Arabic LLMs · Question Answering · Benchmark · Factual verification

Read original article →

Get the weekly digest

Top AI stories from the GCC region, every week.

Related

AraHalluEval: A Fine-grained Hallucination Evaluation Framework for Arabic LLMs

arXiv ·

The paper introduces AraHalluEval, a new framework for evaluating hallucinations in Arabic and multilingual large language models (LLMs). The framework uses 12 fine-grained hallucination indicators across generative question answering and summarization tasks, evaluating 12 LLMs including Arabic-specific, multilingual, and reasoning-based models. Results show factual hallucinations are more common than faithfulness errors, with the Arabic model Allam showing lower hallucination rates. Why it matters: This work addresses a critical gap in Arabic NLP by providing a comprehensive tool for assessing and mitigating hallucination in LLMs, which is essential for reliable AI applications in the Arabic-speaking world.