MBZUAI researchers introduce UniMed-CLIP, a unified Vision-Language Model (VLM) for diverse medical imaging modalities, trained on the new large-scale, open-source UniMed dataset. UniMed comprises over 5.3 million image-text pairs across six modalities: X-ray, CT, MRI, Ultrasound, Pathology, and Fundus, created using LLMs to transform classification datasets into image-text formats. UniMed-CLIP significantly outperforms existing generalist VLMs and matches modality-specific medical VLMs in zero-shot evaluations, improving over BiomedCLIP by +12.61 on average across 21 datasets while using 3x less training data.
MBZUAI researchers introduce XrayGPT, a conversational medical vision-language model for analyzing chest radiographs and answering open-ended questions. The model aligns a medical visual encoder (MedClip) with a fine-tuned large language model (Vicuna) using a linear transformation. To enhance performance, the LLM was fine-tuned using 217k interactive summaries generated from radiology reports.
MBZUAI Ph.D. student Raza Imam and colleagues presented a new benchmark called MediMeta-C to test the robustness of medical vision-language models (MVLMs) under real-world image corruptions. They found that top-performing MVLMs on clean data often fail under mild corruption, with fundoscopy models particularly vulnerable. To address this, they developed RobustMedCLIP (RMC), a lightweight defense using few-shot LoRA tuning to improve model robustness. Why it matters: This research highlights the critical need for robustness testing in medical AI to ensure reliability in clinical settings, particularly in resource-constrained environments where image quality may be compromised.