Skip to content
GCC AI Research

Search

Results for "Preference dataset"

Aligning Cultural Behaviors in Arabic Language Models via Preferences

arXiv ·

This research addresses the underexplored area of aligning Arabic Large Language Models (LLMs) with country-specific cultural behavioral preferences. The authors propose ArabPref, the first large-scale pan-Arab preference dataset designed for cultural behavior alignment conditioned on country. ArabPref includes both a culturally grounded preference dataset for training and evaluation, and a multiple-choice benchmark for assessing culturally aligned behavior across various Arab nations, with all test data validated by native speakers. Why it matters: This dataset and benchmark provide a crucial foundation for systematically studying and improving the cultural nuance and appropriateness of LLMs for the diverse Arab world.

Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMs

arXiv ·

A new culturally inclusive and linguistically diverse dataset called Palm for Arabic LLMs is introduced, covering 22 Arab countries and featuring instructions in both Modern Standard Arabic (MSA) and dialectal Arabic (DA) across 20 topics. The dataset was built through a year-long community-driven project involving 44 researchers from across the Arab world. Evaluation of frontier LLMs using the dataset reveals limitations in cultural and dialectal understanding, with some countries being better represented than others.