Dr. Hao Dong from Peking University presented research on addressing the challenge of limited large-scale training data in embodied AI, particularly for manipulation, task planning, and navigation. The presentation covered simulation learning and large models. Dr. Dong is a chief scientist of China's National Key Research and Development Program and an area chair/associate editor for NeurIPS, CVPR, AAAI, and ICRA. Why it matters: Overcoming data scarcity is crucial for advancing embodied AI research and enabling more sophisticated robotic applications in the region.
Krishna Murthy, a postdoc at MIT, researches computational world models to enable robots to understand and operate effectively in the physical world. His work focuses on differentiable computing approaches for spatial perception and interfaces large image, language, and audio models with 3D scenes. Murthy envisions structured world models working with scaling-based approaches to create versatile robot perception and planning algorithms. Why it matters: This research could significantly advance robotics by enabling more sophisticated perception, reasoning, and action capabilities in embodied agents.
This seminar explores vision systems through self-supervised representation learning, addressing challenges and solutions in mainstream vision self-supervised learning methods. It discusses developing versatile representations across modalities, tasks, and architectures to propel the evolution of the vision foundation model. Tong Zhang from EPFL, with a background from Beihang University, New York University, and Australian National University, will lead the talk. Why it matters: Advancing vision foundation models is crucial for expanding AI applications, especially in the Middle East where computer vision can address challenges in areas like urban planning, agriculture, and environmental monitoring.
Ekaterina Radionova from Smarter AI (formerly Samsung AI Center) presented an approach to generating lifelike real-time avatars. The work focuses on generating high-quality video with authentic facial features to support online generation. Radionova's master's degree is from Skoltech on Data Science program and Bachelor degree at Moscow Institute of Physics and Technology on Applied Math. Why it matters: Achieving realistic real-time avatars is critical for applications in online communication, entertainment, and virtual reality within the region.
Michael Kampffmeyer from UiT The Arctic University of Norway presented a talk at MBZUAI on representation learning for deep clustering and few-shot learning. The talk covered deep clustering in multi-view settings and the influence of geometrical representation properties on few-shot classification performance. He specifically discussed embedding representations on the hypersphere and its connection to the hubness phenomenon. Why it matters: This highlights MBZUAI's role in hosting discussions on advanced machine learning topics like few-shot learning, which are crucial for addressing data scarcity challenges in the region and beyond.
IBM Fellow Dr. Tanveer Syeda-Mahmood gave a talk on the evolution of foundational models, covering multimodal fusion in healthcare and neuro-inspired AI for computer vision. She also discussed image-driven fact-checking of generative AI textual reports for responsible models. Dr. Syeda-Mahmood leads IBM's work in Multimodal Bioinspired AI and WatsonX features, and previously led the Medical Sieve Radiology Grand Challenge. Why it matters: The talk highlights the ongoing development and application of AI foundational models in critical areas like healthcare and responsible AI development, showing IBM's continued investment in these areas.
Researchers at National Taiwan University are developing low-complexity neural network technologies using quantization to reduce model size while maintaining accuracy. Their work includes binary-weighted CNNs and transformers, along with a neural architecture search scheme (TPC-NAS) applied to image recognition, object detection, and NLP tasks. They have also built a PE-based CNN/transformer hardware accelerator in Xilinx FPGA SoC with a PyTorch-based software framework. Why it matters: This research provides practical methods for deploying efficient deep learning models on resource-constrained hardware, potentially enabling broader adoption of AI in embedded systems and edge devices.
This article discusses the evolution of mobile extended reality (MEX) and its potential to revolutionize urban interaction. It highlights the convergence of augmented and virtual reality technologies for mobile usage. A novel approach to 3D models, characterized as urban situated models or “3D-plus-time” (4D.City), is introduced. Why it matters: The development of MEX and 4D.City could significantly enhance user experience and analog-digital convergence in urban environments, offering new possibilities for human-computer interaction.
The article discusses research on fine-tuning text-to-image diffusion models, including reward function training, online reinforcement learning (RL) fine-tuning, and addressing reward over-optimization. A Text-Image Alignment Assessment (TIA2) benchmark is introduced to study reward over-optimization. TextNorm, a method for confidence calibration in reward models, is presented to reduce over-optimization risks. Why it matters: Improving the alignment and fidelity of text-to-image models is crucial for generating high-quality content, and addressing over-optimization enhances the reliability of these models in creative applications.
Nicu Sebe from the University of Trento presented recent work on video generation, focusing on animating objects in a source image using external information like labels, driving videos, or text. He introduced a Learnable Game Engine (LGE) trained from monocular annotated videos, which maintains states of scenes, objects, and agents to render controllable viewpoints. Why it matters: This talk highlights advancements in cross-modal AI, potentially enabling new applications in gaming, simulation, and content creation within the region.
Mingyu Ding from UC Berkeley presented research on endowing robots with human-like commonsense and physical reasoning capabilities. The talk covered multimodal commonsense reasoning integrating vision, world models, and language-based task planners. It also discussed physical reasoning approaches for robots to infer dynamics and physical properties of objects. Why it matters: Enhancing robots with these capabilities can improve their ability to generalize across everyday tasks, leading to greater social benefits and impact.
Egor Zakharov from ETH Zurich AIT lab will present research on creating controllable and detailed 3D head avatars using data from consumer-grade devices. The presentation will cover high-fidelity image-based facial reconstruction/animation and video-based reconstruction of detailed structures like hairstyles. He will showcase integrating human-centric assets into virtual environments for real-time telepresence and entertainment. Why it matters: This research contributes to advancements in digital human modeling and telepresence, with applications in communication and gaming within the region.
Qingbiao Li from the Oxford Robotics Institute is researching decentralized multi-robot coordination using Graph Neural Networks (GNNs). The approach builds an information-sharing mechanism within a decentralized multi-robot system through GNNs and imitation learning. It also uses visual machine learning-assisted navigation with panoramic cameras to guide robots in unseen environments. Why it matters: This research could improve the effectiveness of automated mobile robot systems in urban rail transit and warehousing logistics in the GCC region, where smart city initiatives are growing.
Jorge Amador, a PhD student at KAUST's Visual Computing Center, presented a talk on physically-based simulation for generative AI models. The talk covered the use of synthetic data generation and physical priors to address the need for high-quality datasets. Applications discussed include photo editing, navigation, digital humans, and cosmological simulations. Why it matters: This research explores a promising technique to overcome data scarcity issues in AI, particularly relevant in resource-constrained environments or for sensitive applications.
Pascal Fua from EPFL presented an approach to implementing convolutional neural nets that output complex 3D surface meshes. The method overcomes limitations in converting implicit representations to explicit surface representations. Applications include single view reconstruction, physically-driven shape optimization, and bio-medical image segmentation. Why it matters: This research advances geometric deep learning by enabling end-to-end trainable models for 3D surface mesh generation, with potential impact on various applications in computer vision and biomedical imaging in the region.
This article discusses domain shift in machine learning, where testing data differs from training data, and methods to mitigate it via domain adaptation and generalization. Domain adaptation uses labeled source data and unlabeled target data. Domain generalization uses labeled data from single or multiple source domains to generalize to unseen target domains. Why it matters: Research in mitigating domain shift enhances the robustness and applicability of AI models in diverse real-world scenarios.
Ivan Laptev from INRIA Paris presented a talk at MBZUAI on embodied multi-modal visual understanding, covering advancements in video understanding tasks like question answering and captioning. The talk highlighted recent work on vision-language navigation and manipulation. He argued that detailed understanding of the physical world through vision is still in early stages, discussing open research directions related to robotics and video generation. Why it matters: The discussion of robotics applications and future research directions in embodied AI could influence the direction of AI research and development in the UAE, particularly at MBZUAI.
Axel Sauer from the University of Tübingen presented research on scaling Generative Adversarial Networks (GANs) using pretrained representations. The work explores shaping GANs into causal structures, training them up to 40 times faster, and achieving state-of-the-art image synthesis. The presentation mentions "Counterfactual Generative Networks", "Projected GANs", "StyleGAN-XL”, and “StyleGAN-T". Why it matters: Scaling GANs and improving their training efficiency is crucial for advancing image and video synthesis, with implications for various applications in computer vision, graphics, and robotics.
KAUST Professor Wolfgang Heidrich is researching computational imaging systems that jointly design optics and image reconstruction algorithms. He focuses on hardware-software co-design for imaging systems with applications in HDR, compact cameras, and hyperspectral imaging. Heidrich's work on HDR displays was the basis for Brightside Technologies, acquired by Dolby in 2007. Why it matters: This research aims to advance imaging technology through AI-driven design, potentially impacting various fields from consumer electronics to scientific research within the region and globally.
MBZUAI students won an award at the Asian Conference on Computer Vision (ACCV) for their ObjectCompose method. ObjectCompose generates object-to-background variations of images to validate neural network performance. It helps developers test AI systems by adding variability to validation datasets without distorting the main object. Why it matters: This research offers a new approach to improve the robustness and reliability of computer vision models, which is crucial for real-world applications in the region.
MBZUAI visiting professor of computer vision, Xiaojun Chang, has been named to Clarivate’s 2024 Highly Cited Researchers list, placing him in the top 1% of researchers worldwide. Chang has accumulated over 18,000 citations from around 200 papers, with his research focusing on multimodal foundation models and their applications to embodied AI and healthcare. Chang also holds positions at the University of Technology Sydney and RMIT University in Australia. Why it matters: This recognition highlights MBZUAI's growing impact and its role in fostering interdisciplinary collaboration and innovation in AI research, attracting further opportunities and collaborations.
Nutrigenics, an AI startup founded by MBZUAI PhD students, is developing a platform to improve dietetics. The platform features 200,000 tagged recipes and a vision language model (VLM) that analyzes meal photos with 70% accuracy. Nutrigenics aims to bridge the gap between clinical advice and daily eating habits, enabling dieticians to monitor patient progress more effectively. Why it matters: This technology can personalize nutrition at scale in the region and improve adherence to dietary guidelines, addressing a critical need in preventative healthcare.
Researchers at MBZUAI have developed a new machine learning method called survival rank-n-contrast (SurvRNC) to improve survival models for cancer prognoses. The method is designed to predict survival times for head and neck cancer patients using multimodal data while accounting for censored data (missing values). Numan Saeed presented the team’s work at the 27th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI). Why it matters: Accurate prognoses can significantly improve patient outcomes, and this research contributes to advancements in machine learning techniques for handling complex and incomplete medical data.
MBZUAI will present its latest AI research applications at GITEX 2024, focusing on computer vision, machine learning, NLP, and robotics. The university aims to demonstrate AI's potential in transforming sectors like healthcare, industry, and media, aligning with the UAE’s vision for a knowledge-based economy. MBZUAI will also showcase AI tools designed to combat misinformation and enhance digital media reliability. Why it matters: MBZUAI's presence at GITEX highlights the UAE's commitment to AI innovation and the university's role in developing AI solutions for real-world challenges.
MBZUAI M.Sc. graduate Shamma Al Saedi won the Ministry of Education Entrepreneurship Challenge prize for her startup, Limb, an AI-powered physiotherapy system. The Limb app uses a personalized 3D model and pose estimation to correct patient posture and provide feedback, while the Limb API allows physiotherapists to monitor patient progress. Al Saedi credits MBZUAI's Incubation and Entrepreneurship Center (IEC) for providing knowledge, access to talks, and networking opportunities that helped her develop the startup. Why it matters: This highlights the potential of AI and entrepreneurship programs within UAE universities to foster innovative healthcare solutions and support local talent.
MBZUAI alumnus Adnan Khan is pursuing a Ph.D. at Carleton University, focusing his research on using computer vision to improve accessibility in healthcare, particularly for the visually impaired. His work builds upon his master's thesis at MBZUAI, which focused on domain generalization, enabling models to adapt across different data domains. Khan credits his experiences at MBZUAI for shaping his community spirit and career path. Why it matters: This highlights the role of AI education in fostering socially impactful research and driving innovation in healthcare accessibility in the region and beyond.
MBZUAI Professor Ivan Laptev is working to bridge the gap between data-driven AI systems and embodied agents (robots). He notes challenges in robotics including data scarcity, the need to generate new data through actions, and the requirement for real-time operation. Laptev aims to transfer innovations from computer vision to robotics, addressing these challenges to improve robots' ability to interpret and respond to the complexities of the real world. Why it matters: Overcoming these hurdles is crucial for advancing robotics and enabling robots to effectively interact with and navigate dynamic real-world environments.
MBZUAI has placed over 90 master's and PhD students in internships at public, private, and multinational organizations globally. The internships span various AI disciplines, including computer science, CV, ML, NLP, and robotics. Students are undertaking placements at institutions like the Technical University of Darmstadt, Johns Hopkins University, and Microsoft Research Lab – Asia, as well as UAE entities like ADNOC and Dubai Police. Why it matters: This initiative strengthens the UAE's AI ecosystem by providing hands-on experience for students and addressing the growing demand for AI expertise in the region.
MBZUAI's Metaverse Center is developing technologies for realistic avatar generation. Hao Li and colleagues presented a novel approach at CVPR 2024, collaborating with ETH Zurich, VinAI Research, and Pinscreen. The technology addresses the challenge of mapping 2D images to 3D avatars, accounting for poses, expressions, and views. Why it matters: Creating realistic and efficient avatar generation could improve user experience and accessibility in virtual environments across the Middle East.
MBZUAI's first Ph.D. graduate, Numan Saeed, developed deep learning models to diagnose head and neck cancers using PET and CT scan imagery. His research focused on improving early detection and accurate localization of tumors, aiming to enhance diagnosis and prognosis. Early diagnosis can reduce mortality rates by up to 70%. Why it matters: This research showcases the potential of AI in healthcare to improve cancer diagnosis and treatment, addressing a critical need in resource-constrained healthcare systems.
MBZUAI student Fatima Ahmed Khalil Mohamed Alkhoori is researching machine learning techniques to improve traffic sign recognition for autonomous vehicles. Her work focuses on using transformer model architectures to enhance the ability of autonomous vehicles to accurately recognize traffic signs in varying environmental conditions. The research aims to address challenges such as viewing angle, lighting variations, and shadows that can confuse regular models. Why it matters: This research contributes to the advancement of safe and effective autonomous vehicle navigation, aligning with the UAE's vision of having a world-class transportation system.
MBZUAI graduate Ahmed Sharshar developed a computer vision application that assesses lung health from a video of a person breathing, estimating Forced Vital Capacity (FVC), Forced Expiratory Volume in 1 second (FEV1), and Peak Expiratory Flow (PEF). The model achieved up to 100% accuracy using thermal video data from 60 participants. Sharshar aims to create lightweight models applicable in developing countries without high-end GPUs. Why it matters: This research showcases the potential of AI to democratize healthcare access through non-invasive, accessible diagnostic tools.
MBZUAI computer vision graduate Yahia Dalbah pivoted from electrical engineering to a role at FortyGuard, a clean tech startup focused on geospatial analysis of outdoor temperatures. He will be a research and development engineer, responsible for machine learning and computer vision algorithms. FortyGuard provides governments and businesses with tools to measure and analyze factors affecting heat and temperature. Why it matters: This highlights the potential for AI education in the UAE to drive innovation in sustainability and environmental solutions through local startups.
MBZUAI researchers are developing an AI-powered tool for flood assessment using satellite data and computer vision, prompted by the recent extreme weather in the Gulf region. The prototype analyzes spatial satellite imagery from before and after the storm to detect changes and identify heavily impacted roads and critical infrastructure. The tool uses AI models, Sentinel-2 imagery, and OpenStreetMap data to locate affected areas and estimate water depth. Why it matters: This research offers a way to automate and improve rapid response to extreme weather events, providing local authorities with critical information for rescue, recovery, and future urban planning in the face of climate change.
MBZUAI Professor Ian Reid discusses his career in embodied AI, from early work on active vision at Oxford to current research. He highlights three key developments: cameras as geometric sensors, visual SLAM, and advancements in robot navigation. Reid distinguishes embodied AI from systems like ChatGPT, emphasizing its need for understanding and interaction with the physical world. Why it matters: The insights from a leading expert underscore the importance of embodied AI as the next frontier in intelligent systems and robotics in the region.
The first International Olympiad in AI for high school students will be held in Bulgaria from August 9-15, 2024. Organized by the LERAI Foundation, the competition will test students on machine learning, computer vision, and natural language processing. Sponsors include Google and MBZUAI, with organizers hoping participants pursue further AI education and careers. Why it matters: This event aims to cultivate global AI talent and could increase interest in AI education and careers in the GCC region, particularly at MBZUAI.
MBZUAI researchers are developing AI applications for malaria prevention in Indonesia using sensory data fusion and digital twins. Another MBZUAI team is using machine learning and computer vision to detect cardiovascular disease from CT scans in collaboration with the University of Oxford. AI-powered remote patient monitoring is also being explored for proactive interventions and chronic disease management. Why it matters: These projects demonstrate the potential of AI to address healthcare challenges in underserved communities and improve disease prevention and management in the region.
MBZUAI researchers have developed a new approach to enhance the generalizability of vision-language models when processing out-of-distribution data. The study, led by Sheng Zhang and involving multiple MBZUAI professors and researchers, addresses the challenge of AI applications needing to manage unforeseen circumstances. The new method aims to improve how these models, which combine natural language processing and computer vision, handle new information not used during training. Why it matters: Improving the adaptability of vision-language models is critical for real-world AI applications like autonomous driving and medical imaging, especially in diverse and changing environments.
MBZUAI's Dr. Hao Li is working on using AI and 3D telepresence to transform communication, work, and education by replacing physical transportation with virtual teleportation. His research focuses on the intersection of computer graphics, computer vision, and AI, specifically virtual avatar creation and facial performance capture. Li aims to improve communication using AI to achieve what cannot be done in real life. Why it matters: This research has the potential to reduce carbon footprints by enabling remote work and virtual collaboration, while also positioning MBZUAI and the UAE as leaders in AI-driven metaverse technologies.
MBZUAI awarded its first Ph.D. to Numan Saeed for his dissertation, "Deep Learning for Cancer Diagnosis and Prognosis." Saeed's research explores using deep learning models to diagnose tumors and predict survival rates for head and neck cancers by interpreting PET and CT scan imagery and doctor's notes. The AI model considers variables like age, gender, and tumor size to improve diagnosis, especially in regions with limited oncological services. Why it matters: This milestone highlights MBZUAI's growing role in AI research and its potential to contribute to advancements in AI-driven healthcare solutions, particularly for cancer diagnosis and treatment.
A team of MBZUAI graduate students won first place at the UAE University's University Challenge for their SawabAI project, which addresses AI-generated misinformation about climate change. The winning team included Salem Bin Saqer AlMarri, Hanoona Bangalath, and Muhammad Maaz, all Computer Vision Ph.D. students. SawabAI is envisioned as a platform to evaluate the authenticity and source of information, including text, image, and video, to combat fake news. Why it matters: This win highlights the growing importance of AI in addressing misinformation and promoting sustainability in government communication within the region.
MBZUAI researchers presented a new approach to video question answering at ICCV 2023. The method leverages insights from analyzing still images to understand video content, potentially reducing the computational resources needed for training video question answering models. Guangyi Chen, Kun Zhang, and colleagues aim to apply pre-trained image models to understand video concepts. Why it matters: This research could lead to more efficient and accessible video analysis tools, benefiting fields like healthcare and security where video data is abundant.
MBZUAI had 30 papers accepted at the International Conference on Computer Vision (ICCV) in Paris, out of 8,260 submissions. Visiting Professor Ivan Laptev served as one of the ICCV Program Chairs. Two papers from MBZUAI researchers focused on analyzing moving images, with one introducing Video-FocalNets for action analysis and the other exploring the transfer of knowledge from still image analysis to video. Why it matters: MBZUAI's strong presence at ICCV demonstrates its growing prominence in the global computer vision research landscape.
MBZUAI researchers presented a new approach to video analysis at ICCV in Paris, led by Syed Talal Wasim. The approach builds on still image processing techniques like focal modulation to analyze spatial and temporal information in video separately. It aims to improve temporal aggregation while avoiding the computational complexity of transformers. Why it matters: This research advances video understanding in computer vision by offering a more efficient method for temporal modeling, crucial for applications like activity recognition and video surveillance.
An MBZUAI team developed a self-ensembling vision transformer to enhance the security of AI in medical imaging. The model aims to protect patient anonymity and ensure the validity of medical image analysis. It addresses vulnerabilities where AI systems can be manipulated, leading to misinterpretations with potentially harmful consequences in healthcare. Why it matters: This research is crucial for building trust and enabling the safe deployment of AI in sensitive medical applications, protecting against fraud and ensuring patient safety.
Researchers at MBZUAI developed a method to measure vital signs using webcams by analyzing color intensity changes in facial blood flow. They built a digital twin system that uses machine learning to combine heart rate, respiratory rate, and blood oxygen level measures. The system displays real-time vital sign information, enabling remote patient triage. Why it matters: This research contributes to the advancement of telemedicine, potentially improving healthcare access in underserved regions and aligning with UN Sustainable Development Goal #3.
MBZUAI graduate Ahmed Elhagry received the Best Ph.D. Forum Paper Award at IEEE MetaCom 2023 for his work on generating virtual worlds using digital twin technology and AI through a text-to-metaverse pipeline. The research addresses the challenge of manually constructing 3D models for metaverse applications, which is time-consuming and costly. Elhagry's approach uses multimodal AI and DT technology to produce realistic virtual environments, benefiting industries like education, healthcare, and entertainment. Why it matters: This award recognizes innovative research at MBZUAI with the potential to advance metaverse development and AI-driven virtual environment creation in the region.
MBZUAI welcomed 142 students from 34 countries, its largest and most diverse cohort since 2019, with a competitive admission rate of 4.6% out of over 4,000 applications. The new students are pursuing master's and Ph.D. degrees in computer vision, machine learning, and natural language processing, bringing the total student body to nearly 300. MBZUAI also established new robotics and computer science departments to meet global demand. Why it matters: This expansion strengthens the UAE's position as a global AI research hub, attracting top talent and fostering innovation in key AI disciplines.
MBZUAI hosted 34 undergraduate STEM students from around the world for its inaugural one-month Undergraduate Research Internship Program (UGRIP) in Abu Dhabi. The interns, hailing from countries including the UAE, USA, and India, engaged in ongoing MBZUAI research projects spanning sustainability, education, and healthcare. Interns worked on projects ranging from fake news detection using NLP to tumor diagnosis using computer vision. Why it matters: This program strengthens MBZUAI's position as a global AI research hub and cultivates AI talent within the region.
MBZUAI graduate Gokul Karthik Kumar credits the university's interdisciplinary approach for helping him discover his passion for working at the intersection of computer vision and NLP. He chose MBZUAI over the University of Waterloo due to the flexibility to explore different AI domains. Kumar will join G42’s Inception Institute of Artificial Intelligence (IIAI) as an applied scientist to develop large language models tailored for UAE-focused applications. Why it matters: This highlights MBZUAI's role in nurturing AI talent for the UAE and G42's focus on developing local LLMs.