Skip to content
GCC AI Research

The Qiyas Benchmark: Measuring ChatGPT Mathematical and Language Understanding in Arabic

arXiv · · Significant research

Research NLP Arabic AI LLM Benchmark

Summary

Researchers introduce two new benchmarks, derived from the Qiyas exam, to evaluate mathematical reasoning and language understanding in Arabic. They tested ChatGPT-3.5-turbo and ChatGPT-4, which achieved 49% and 64% accuracy respectively. The new benchmarks aim to address the lack of resources for evaluating Arabic language models.

Keywords

Arabic · Language model · Benchmark · ChatGPT · Qiyas exam

Read original article →

Get the weekly digest

Top AI stories from the GCC region, every week.