MBZUAI, G42, and Cerebras Systems have launched K2 Think V2, a 70-billion parameter reasoning system built on the K2-V2 base model. K2 Think V2 is fully open-source, from pre-training data to post-training alignment, ensuring transparency and reproducibility. It achieves leading results on complex reasoning benchmarks like AIME2025 and GPQA-Diamond. Why it matters: This release marks a significant advancement in the UAE's AI capabilities, demonstrating leadership in building globally accessible and fully sovereign AI systems focused on reasoning.
MBZUAI's Institute of Foundation Models (IFM) has released K2 Think V2, a 70 billion parameter open-source general reasoning model built on K2 V2 Instruct. The model excels in complex reasoning benchmarks like AIME2025 and GPQA-Diamond, and features a low hallucination rate with long context reasoning capabilities. K2 Think V2 is fully sovereign and open, from pre-training through post-training, using IFM-curated data and a Guru dataset. Why it matters: This release contributes to closing the gap between community-owned reproducible AI and proprietary models, particularly in reasoning and long-context understanding for Arabic NLP tasks.
MBZUAI researchers presented a method for cross-cultural transfer learning to improve language models' understanding of diverse Arab cultures. They used in-context learning and demonstration-based reinforcement (DITTO) to transfer cultural knowledge between countries. Experiments showed up to 34% improvement in performance on cultural understanding benchmarks using only a few demonstrations. Why it matters: This research addresses the gap in cultural understanding of Arabic language models, especially for smaller Arab countries, and provides a novel transfer learning approach.
Researchers from MBZUAI, Carnegie Mellon University, and Meta AI presented a new approach called ThoughtComm at NeurIPS 2025 where AI agents communicate through internal, latent representations instead of natural language. This framework extracts and selectively shares latent "thoughts" from agents' internal states, representing the underlying structure of their reasoning. Results show that agents coordinate more effectively, reach consensus faster, and solve problems more accurately using this method. Why it matters: Bypassing the limitations of natural language in AI communication could lead to more efficient and accurate multi-agent systems, impacting areas like robotics, collaborative AI, and distributed problem-solving.
Inception, Cerebras, and MBZUAI have released Jais 2, a 70 billion parameter open-weight Arabic LLM. Jais 2 is trained on an Arabic-first dataset and features a redesigned architecture for stronger reasoning and fluency across Arabic dialects and English. It integrates a safety-first framework and demonstrates capabilities in understanding Arabic poetry, culture, and social media tone. Why it matters: Jais 2 addresses the historical underrepresentation of Arabic in AI by providing a culturally and linguistically faithful model, potentially accelerating innovation across the region.
MBZUAI's Institute of Foundation Models has released K2, a 70-billion-parameter, reasoning-centric foundation model. K2 is designed to be fully inspectable, with open weights, training code, data composition, mid-training checkpoints, and evaluation harnesses. K2 outperforms Qwen2.5-72B and approaches the performance of Qwen3-235B. Why it matters: This release promotes transparency and reproducibility in AI development, providing researchers with the resources needed to study, adapt, and build upon a strong foundation model.
MBZUAI researchers have introduced SURPRISE3D, a benchmark for evaluating 3D spatial reasoning in AI systems, along with a 3D Spatial Reasoning Segmentation (3D-SRS) task. The benchmark includes over 900 indoor scenes and 200,000 language queries paired with 3D masks, emphasizing spatial relationships over object naming. A companion paper, MLLM-For3D, explores adapting 2D multimodal LLMs for 3D reasoning. Why it matters: This work addresses a key limitation in current AI, pushing towards embodied AI that can understand and act in 3D environments based on human-like spatial reasoning.
MBZUAI researchers at the Institute of Foundation Models (IFM) investigated the role of reinforcement learning (RL) in improving reasoning abilities of language models. Their study found that RL acts as an 'elicitor' for reasoning in domains frequently encountered during pre-training (e.g., math, coding), while genuinely teaching new reasoning skills in underrepresented domains (e.g., logic, simulations). To support their analysis, they created a new dataset called GURU containing 92,000 examples across six domains. Why it matters: This research clarifies the impact of reinforcement learning on language model reasoning, paving the way for developing models with more generalizable reasoning abilities across diverse domains, an important direction for more capable AI systems.
A new paper from MBZUAI demonstrates that state-of-the-art speech models can be easily jailbroken using audio perturbations to generate harmful content, achieving success rates of 76-93% on models like Qwen2-Audio and LLaMA-Omni. The researchers adapted projected gradient descent (PGD) to the audio domain to optimize waveforms that push the model towards harmful responses. They propose a defense mechanism based on post-hoc activation patching that hardens models at inference time without retraining. Why it matters: This research highlights a critical vulnerability in speech-based LLMs and offers a practical solution, contributing to the development of more secure and trustworthy AI systems in the region and globally.
Researchers from MBZUAI, IBM, and ServiceNow introduced GEOBench-VLM, a benchmark for evaluating vision-language models on Earth observation tasks using satellite and aerial imagery. The benchmark includes over 10,000 human-verified instructions across 31 sub-tasks spanning object classification, localization, change detection, and more. GEOBench-VLM addresses the gap in current VLMs' ability to perform spatially grounded reasoning and change detection in satellite imagery. Why it matters: This benchmark will drive progress in AI's ability to analyze satellite data for critical applications like disaster response, climate monitoring, and urban planning in the Middle East and globally.
MBZUAI and G42 have launched K2 Think, an open-source AI system for advanced reasoning with 32 billion parameters. It outperforms reasoning models 20 times larger, employing techniques like long chain-of-thought fine-tuning and reinforcement learning. K2 Think will be available on Cerebras' platform, achieving 2,000 tokens per second, and ranks highly in math performance. Why it matters: This launch positions the UAE as a leader in AI innovation through public-private partnerships and open-source contributions, demonstrating that efficient AI design can rival larger models.
Researchers at MBZUAI have developed a new automatic method to examine cross-lingual abilities in multilingual language models, testing 10 models across 16 languages. They combined beam search with language-model-based simulation, generating 6,000 bilingual question pairs and found significant performance drops compared to English, even in high-resource languages like Chinese. The method introduces perturbations to test the models' ability to transfer knowledge rather than rely on memorization. Why it matters: This research highlights critical gaps in cross-lingual AI, providing a framework for developing more equitable and effective multilingual models, especially for Arabic and other under-represented languages.
MBZUAI researchers developed AirCast, a novel AI model for improved air pollution forecasting, which won the best paper award at the TerraBytes workshop during ICML. AirCast fuses weather and chemistry data using a Vision Transformer and frequency-weighted MAE to better predict extreme events like Saharan dust storms. In tests across the Middle East and North Africa, AirCast reduced PM2.5 error by 33% compared to a persistence baseline and outperformed the CAMS physics model. Why it matters: Accurate air pollution forecasting is critical for public health in the GCC region, and this research demonstrates a significant advancement using AI to address this challenge.
MBZUAI researchers, in collaboration with Monash University, have introduced ArEnAV, a new dataset for deepfake detection featuring Arabic-English code-switching. The dataset comprises 765 hours of manipulated YouTube videos, incorporating intra-utterance code-switching and dialect variations. Experiments showed that code-switching significantly reduces the performance of existing deepfake detectors. Why it matters: This work addresses a critical gap in AI's ability to handle linguistic diversity, particularly in regions where code-switching is prevalent, enhancing the reliability of deepfake detection in real-world scenarios.
Researchers at MBZUAI and other institutions have published a study at ACL 2024 investigating how jailbreak attacks work on LLMs. The study used a dataset of 30,000 prompts and non-linear probing to interpret the effects of jailbreak attacks, finding that existing interpretations were inadequate. The researchers propose a new approach to improve LLM safety against such attacks by identifying the layers in neural networks where the behavior occurs. Why it matters: Understanding and mitigating jailbreak attacks is crucial for ensuring the responsible and secure deployment of LLMs, particularly in the Arabic-speaking world where these models are increasingly being used.
MBZUAI researchers have created ArabCulture, a new benchmark dataset to measure cultural commonsense reasoning capabilities in Arabic language models. The dataset was built by native Arabic speakers from 13 countries and is the largest of its kind. Testing 31 language models, the researchers found that many systems struggle with understanding cultural concepts across the Arab world. Why it matters: The new benchmark addresses a gap in AI, enabling development of culturally-aware AI systems tailored to the nuances of the Arabic-speaking world.
MBZUAI researchers Nils Lukas and Toluwani Samuel Aremu will present a paper at ICML 2025 demonstrating the vulnerability of current watermarking techniques in LLMs. Their research shows that adaptive paraphrasers can evade detection from watermarks with negligible impact on text quality, costing less than $10 of GPU compute. The attack involves fine-tuning a small open-weight model to rewrite sentences until surrogate keys no longer trigger detection. Why it matters: This work highlights critical weaknesses in current AI provenance methods, suggesting the need for more robust watermarking techniques to maintain trust in the authenticity of AI-generated content.
MBZUAI is conducting research to improve cross-cultural understanding using AI, including studying LLM limitations in recognizing cultural references. They developed "Culturally Yours," a tool that helps users comprehend cultural references in text, and the "All Languages Matter Benchmark" (ALM Bench) to evaluate multimodal LLMs across 100 languages. MBZUAI has also developed LLMs tailored to low-resource languages like Jais (Arabic), Nanda (Hindi), and Sherkala (Kazakh). Why it matters: These initiatives promote inclusivity and ensure AI systems are culturally aware and can serve diverse populations effectively, particularly in the Middle East's multicultural context.
MBZUAI researchers presented FIRE, a new fact-checking framework for LLM outputs, at NAACL 2025. FIRE first assesses the LLM's confidence in its claims before searching the web, reducing computational cost. It also stores knowledge gained from web searches to aid in classifying other claims. Why it matters: This approach improves the efficiency and cost-effectiveness of automatically verifying the accuracy of LLMs, addressing a key limitation in their reliability.
MBZUAI and École Polytechnique are deepening research collaboration through a Collaborative Research Agreement, focusing on large language models, foundation models for reasoning, and AI applications in biology, health, and AI safety. The partnership builds on a previous MoU and Scholars Exchange Program Agreement between the two institutions. MBZUAI's France Lab has developed Atlas-Chat, a family of open-source LLMs for the Moroccan Arabic dialect Darija, with models including Atlas-Chat-2B and Atlas-Chat-9B. Why it matters: This collaboration strengthens the AI ecosystems in both France and the UAE, fostering joint research efforts and supporting the next generation of AI researchers and innovators, with a specific focus on Arabic NLP.
MBZUAI researchers have developed a new benchmark for evaluating the teaching abilities of large language models (LLMs), earning the SAC Award for Resources and Evaluation at NAACL 2025. The framework aims to measure how effectively LLMs can be used for personalized tutoring, addressing the "two sigma problem" in education. Unlike rule-based tutoring systems, LLMs offer fluency but lack pedagogical principles. Why it matters: This benchmark is a crucial step towards integrating learning science into AI, potentially enabling personalized AI tutors that significantly improve educational outcomes.
MBZUAI researchers created a new benchmark dataset called TextGames to evaluate the reasoning abilities of LLMs. The dataset uses simple, text-based games requiring skills like pattern recognition and logical thinking. LLMs struggled with the hardest questions, suggesting limitations in their reasoning capabilities despite advancements in language understanding. Why it matters: This research highlights the need for specialized reasoning models and benchmarks that go beyond memorization to truly test AI's problem-solving abilities.
MBZUAI researchers, in collaboration with over 70 researchers, have created the Culturally diverse Visual Question Answering (CVQA) benchmark to evaluate cultural understanding in multimodal LLMs. The CVQA dataset includes over 10,000 questions in 31 languages and 13 scripts, testing models on images of local dishes, personalities, and monuments. Testing of several multimodal LLMs on the CVQA benchmark revealed significant challenges, even for top models. Why it matters: This benchmark highlights the need for AI models to better understand diverse cultures, promoting fairness and relevance across different languages and regions.
The 31st International Conference on Computational Linguistics (COLING 2025) is being held in Abu Dhabi from January 18-24, hosted by MBZUAI. The conference features paper presentations, demonstrations, keynote speeches, workshops, and tutorials, with over 1,500 attendees. MBZUAI faculty and students contributed 22 papers to the conference, including research on fact-checking and cross-cultural content. Why it matters: Hosting COLING 2025 highlights the UAE's growing role as a hub for AI and NLP research, particularly in Arabic language processing.
MBZUAI researchers have released ALM Bench, a new benchmark dataset for evaluating the performance of multimodal LLMs on cultural visual question-answer tasks across 100 languages. The dataset includes over 22,000 question-answer pairs across 19 categories, with a focus on low-resource languages and cultural nuances, including three Arabic dialects. They tested 16 open- and closed-source multimodal LLMs on it, revealing a significant need for greater cultural and linguistic inclusivity. Why it matters: The benchmark aims to improve the inclusivity of multimodal AI systems by addressing the underrepresentation of low-resource languages and cultural contexts.
This article discusses MBZUAI's efforts in advancing Arabic language AI, including the development of advanced linguistic models using deep learning techniques. Key initiatives include Jais, a 13B parameter Arabic LLM developed in collaboration with G42's Inception, and Atlas-Chat, which understands the Moroccan dialect. The university is also incorporating Arabic in practical AI solutions like BiMediX2, a healthcare multi-modal model that understands medical queries in both English and Arabic. Why it matters: These initiatives are crucial for preserving Arabic cultural heritage, enabling future discovery, and addressing linguistic challenges specific to the Arabic language in AI applications.
A study co-authored by researchers from UC Berkeley, University of the Witwatersrand, Lelapa AI, and MBZUAI received the Outstanding Paper Award at EMNLP 2024. The paper critiques the term "low-resource" languages in NLP, highlighting its limitations in capturing the diverse challenges faced by different languages. The authors propose a more detailed analysis of resourcedness to encourage targeted support for languages currently underserved by technology. Why it matters: The research challenges assumptions in NLP and promotes more nuanced approaches to supporting the world's many languages, including Arabic, in AI systems.
MBZUAI doctoral student Mai A. Shaaban and colleagues developed MedPromptX, a system that analyzes chest X-rays and patient data to aid lung disease diagnoses. MedPromptX uses multimodal large language models with visual grounding and few-shot prompting, trained on a new dataset of 6,000 patient records (MedPromptX-VQA) derived from MIMIC-IV and MIMIC-CXR. The system addresses the challenge of incomplete electronic health records by leveraging the knowledge embedded in large language models to interpret lab results. Why it matters: This research advances AI-driven medical diagnostics by integrating diverse data sources and addressing data gaps, potentially leading to quicker and more accurate diagnoses.
MBZUAI researchers developed a machine-learning method to predict antimicrobial resistance (AMR) by analyzing electronic health records. The system predicts if a patient will experience AMR when prescribed an antibiotic or if infected with a bacterium. Published in Scientific Reports, the innovation helps physicians identify patients at risk for AMR by using patient demographics, lab results, and physician notes. Why it matters: This approach can help combat the rise of drug-resistant bacteria by providing timely predictions and supporting more informed prescription decisions.
MBZUAI researchers have created ArabicMMLU, the first benchmark dataset in Modern Standard Arabic for evaluating language understanding across multiple tasks. The dataset contains over 14,000 multiple-choice questions from school exams across the Arabic-speaking world and addresses the limitations of translated English datasets. It was presented at the 62nd Annual Meeting of the Association for Computational Linguistics in Bangkok. Why it matters: This benchmark enables a more accurate and culturally relevant evaluation of LLMs' capabilities in Arabic, which is crucial for developing AI tailored to the Arab world.
MBZUAI researchers presented EXAMS-V, a new benchmark dataset for evaluating the reasoning and processing abilities of vision language models (VLMs). EXAMS-V contains over 20,000 multiple-choice questions across 26 subjects and 11 languages, including Arabic. The dataset presents the questions within images, testing the VLM's ability to integrate visual and textual information. Why it matters: This dataset fills a gap in VLM evaluation, providing a valuable resource for assessing and improving the multimodal reasoning capabilities of these models, particularly in diverse languages like Arabic.
A study by MBZUAI's Preslav Nakov and Cornell co-authors examines how to develop systems that detect fake news in a landscape where text is generated by humans and machines. The research, presented at the 2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics, analyzes fake news detectors' ability to identify human- and machine-written content. The study highlights biases in current detectors, which tend to classify machine-written news as fake and human-written news as true. Why it matters: Addressing these biases is crucial as machine-generated content becomes more prevalent in both real and fake news, requiring more nuanced detection methods.
MBZUAI M.Sc. graduate Mohammad Hanan Ghani developed new techniques to improve text-to-image generation from long text prompts, combining large language models and diffusion models. Advised by Dr. Salman Khan, Ghani published three papers at ICLR, BMVC, and NeurIPS, with the ICLR paper focusing on generating images that accurately reflect detailed text descriptions. The new system improves upon existing techniques to generate images that closely follow the details of the input text. Why it matters: This research addresses a key limitation in current T2I models and advances the field of multimodal AI, potentially improving the capabilities of robots and autonomous devices.
MBZUAI, Petuum, and LLM360 have launched K2-65B, an open-source 65B parameter LLM, trained on 1.4T tokens using 480 A100 GPUs. K2-65B outperforms Llama 2 70B while using 35% fewer resources, emphasizing sustainable AI development. The model and its chat variant, K2-Chat, excel in math, coding, medicine, and human-like response generation, with the model available under the Apache 2.0 license. Why it matters: This launch highlights the UAE's increasing capabilities in developing efficient and high-performing LLMs, promoting open-source collaboration and setting new standards for sustainable AI practices in the region.
MBZUAI and the University of Michigan Ann Arbor have announced a new collaboration in AI research, sponsored by the U.S. Mission to the UAE. The partnership focuses on projects addressing the cultural divide in AI, with research teams from both institutions collaborating throughout the 2023/2024 academic year. A workshop titled “Bridging the Cultural Divide in AI: Analyzing Fairness, Bias, and Transparency across Cultures” will be held on April 29-30. Why it matters: The collaboration strengthens ties between the UAE and the U.S. in AI, addressing critical issues of fairness and cultural sensitivity in AI development.
MBZUAI's Institute of Foundation Models (IFM) has launched five new specialized language and multimodal models, including BiMediX, PALO, GLaMM, GeoChat, and MobiLLaMA. These models address real-world applications in healthcare, visual reasoning, multilingual capabilities, geospatial analysis, and mobile device efficiency. BiMediX is a bilingual medical LLM, while GLaMM generates natural language responses related to objects in an image at the pixel level. Why it matters: This launch demonstrates MBZUAI's commitment to advancing AI research and developing practical AI solutions for various industries, especially with a focus on Arabic language capabilities.
MBZUAI increased faculty diversity and worked with global partners on application projects in 2023, including developing Jais (with Core42 and Cerebras) and Vicuna (with UC San Diego, UC Berkeley, CMU, and Stanford). They also launched Jais Climate, a bilingual LLM for climate intelligence, and LLM360, a framework for transparent LLM research. Why it matters: MBZUAI's involvement in open-source GenAI initiatives and partnerships positions the UAE as a key player in responsible AI development and talent creation.
MBZUAI faculty and students will present 44 papers at the Empirical Methods in Natural Language Processing (EMNLP) conference in Singapore. Research topics include disinformation detection, social media analysis, dialogue generation, and Arabic LLMs. Preslav Nakov, Iryna Gurevych, Timothy Baldwin, Alham Fikri Aji, and Muhammad Abdul-Mageed are among the MBZUAI researchers presenting at the conference. Why it matters: MBZUAI's strong presence at a top NLP conference highlights the UAE's growing contributions to cutting-edge AI research and its increasing global prominence in the field.
An MBZUAI team won the best paper award at the inaugural Arabic Natural Language Processing Conference for their work on processing Arabic speech. Their study establishes a new approach to tackle the complexities of spoken Arabic, which differs significantly from text-based language models. The team's approach aims to advance new tools for Arabic speakers by addressing challenges like intonation and the continuous nature of speech. Why it matters: This award highlights the importance of specialized research in Arabic NLP, as mainstream LLMs often face limitations in accurately processing the nuances of Arabic speech.
MBZUAI researchers created Bactrian-X, a new dataset to improve LLM instruction following in low-resource languages. The dataset leverages instruction tuning, pairing instructions in various languages with expected responses. Bactrian-X builds upon existing open-source instruction tuning models. Why it matters: This work aims to democratize access to LLMs by enabling users to interact with them in their native languages, even when English proficiency is limited.
MBZUAI and Core42 have launched Jais Climate, the first bilingual (Arabic/English) LLM dedicated to climate and sustainability. It is fine-tuned with 1.4 million climate-related instructions and trained on the ClimaInstruct dataset. Jais Climate is built on Jais 13B and incorporates technology from Vicuna, an open-source LLM. Why it matters: This model provides accessible climate data to a wide audience, including decision-makers and the general public, and highlights the UAE's focus on using AI for sustainability.
MBZUAI has launched the Institute of Foundation Models to advance generative AI research and development. The institute will focus on developing large-scale AI models adaptable to various applications, building on MBZUAI's existing work with models like Jais, Llama 2, and Vicuna. It will focus on multi-modal foundation models applicable to areas like healthcare, finance and environmental engineering. Why it matters: This initiative further solidifies the UAE's position as a leader in AI, particularly in the development and application of foundation models for diverse industries.
G42's Inception has open-sourced Jais, a 13-billion parameter Arabic large language model (LLM). Jais was trained on a 395-billion-token Arabic and English dataset and outperforms existing Arabic models. The model is a collaboration between Inception, MBZUAI, and Cerebras Systems, and was trained on the Condor Galaxy supercomputer. Why it matters: This release establishes a new standard for Arabic language AI, providing over 400 million Arabic speakers access to generative AI and fostering innovation in the region.
Researchers from MBZUAI, UC Berkeley, CMU, Stanford, and UC San Diego collaborated to create Vicuna, an open-source chatbot that costs $300 to train, unlike ChatGPT which costs over $4 million. Vicuna achieves 90% of ChatGPT's subjective language quality while being far more energy-efficient and can run on a single GPU. It was fine-tuned from Meta AI’s LLaMA model using user-shared conversations and has gained significant traction on GitHub. Why it matters: This research demonstrates that high-quality chatbots can be developed at a fraction of the cost and environmental impact, opening up new possibilities for sustainable AI development in the region.
MBZUAI researchers had 26 papers accepted at ACL 2023, a top NLP conference. Assistant Professor Alham Fikri Aji co-authored eight papers, including one on crosslingual generalization through multitask finetuning (MTF). Deputy Department Chair Preslav Nakov co-authored a paper on a Bulgarian language understanding benchmark dedicated to the memory of Yale Computer Scientist Dragomir R. Radev. Why it matters: MBZUAI's strong presence at ACL highlights its growing influence in the NLP field and its contributions to multilingual AI research.
MBZUAI President Eric Xing led a global collaboration to develop Vicuna, an LLM alternative to GPT-3 addressing the unsustainable costs of training LLMs. OpenAI CEO Sam Altman acknowledged Abu Dhabi's role in the global AI conversation, building off of achievements like Vicuna. Xing and colleagues are publishing research at MLSys 2023 on "cross-mesh resharding" to improve computer communication in deep learning, aiming for low-carbon, affordable, and miniaturized AI. Why it matters: This research signals a push towards sustainable AI development in the region, emphasizing efficiency and reduced environmental impact.
A new paper from MBZUAI researchers explores using ChatGPT to combat the spread of fake news. The researchers, including Preslav Nakov and Liangming Pan, demonstrate that ChatGPT can be used to fact-check published information. Their paper, "Fact-Checking Complex Claims with Program-Guided Reasoning," was accepted at ACL 2023. Why it matters: This research highlights the potential of large language models to address the growing challenge of misinformation, with implications for maintaining information integrity in the digital age.
MBZUAI and IBM have launched an AI Center of Excellence to drive sustainability. The center will focus on developing carbon-neutral energy solutions and improving natural language processing for Arabic dialects. Joint research will focus on decarbonization, climate change mitigation, and foundational AI models to support renewable energy. Why it matters: This initiative addresses critical regional needs in sustainability and Arabic NLP, aligning with the UAE's AI strategy and COP28 goals.
G42 has launched Nanda 87B, an open-source Hindi-English LLM developed by MBZUAI in collaboration with Inception and Cerebras. Nanda 87B is built upon Llama-3.1 70B and trained on a dataset with over 65 billion Hindi tokens. The model is engineered for real-world use being fluent in formal Hindi, casual speech, and Hinglish, and is designed for translation, summarization, instruction-following, and transliteration tasks. Why it matters: This release marks a major advancement in creating inclusive AI technology tailored for one of the world's largest linguistic communities.
MBZUAI researchers release JEEM, a new benchmark dataset for evaluating vision-language models on Arabic dialects. The dataset covers image captioning and visual question answering tasks using images from Jordan, UAE, Egypt, and Morocco. Results show models struggle with cultural understanding and relevance despite fluent language generation.