General AI research publications from MBZUAI, KAUST, TII, QCRI, and other GCC institutions, including conference papers, journal articles, and preprints.
A research talk was given on privacy and security issues in speech processing, highlighting the unique privacy challenges due to the biometric information embedded in speech. The talk covered the legal landscape, proposed solutions like cryptographic and hashing-based methods, and adversarial processing techniques. Dr. Bhiksha Raj from Carnegie Mellon University, an expert in speech and audio processing, delivered the talk. Why it matters: As speech-based interfaces become more prevalent in the Middle East, understanding and addressing the associated privacy risks is crucial for ethical AI development and deployment.
Fudan University's Zhongyu Wei presented research on social simulation driven by LLMs, covering individual and large-scale social movement simulation. Wei directs the Data Intelligence and Social Computing Lab (Fudan DISC) and has published extensively on multimodal large models and social computing. His work includes the Volcano multimodal model, DISC-MedLLM, and ElectionSim. Why it matters: Using LLMs for social simulation could provide new tools for understanding and potentially predicting social dynamics in the Arab world.
Ahmed Elhag, a PhD student at the University of Oxford, presented a new training procedure that approximates equivariance in unconstrained machine learning models via a multitask objective. The approach adds an equivariance loss to unconstrained models, allowing them to learn approximate symmetries without the computational cost of fully equivariant methods. Formulating equivariance as a flexible learning objective allows control over the extent of symmetry enforced, matching the performance of strictly equivariant baselines at a lower cost. Why it matters: This research from a speaker at MBZUAI balances rigorous theory and practical scalability in geometric deep learning, potentially accelerating drug discovery and design.
A presentation discusses the evolution of Vision-and-Language Navigation (VLN) from benchmarks like Room-to-Room (R2R). It highlights the role of Large Language Models (LLMs) such as GPT-4 in enabling more natural human-machine interactions. The presentation showcases work using LLMs to decode navigational instructions and improve robotic navigation. Why it matters: This research demonstrates the potential of merging vision, language, and robotics for advanced AI applications in navigation and human-computer interaction.
A report discusses using AI to optimize healthcare delivery across the entire medical process cycle, including pre-hospital screening, in-hospital treatment, and post-hospital rehabilitation. It considers optimal management of workflow, medical resources, and comprehensive healthcare coverage. Dr. Jingshan Li from Tsinghua University is the author, with extensive publications and experience in production and healthcare systems. Why it matters: AI-driven improvements to healthcare processes could lead to better resource allocation and enhanced patient outcomes across the GCC region.
A Caltech researcher presented at MBZUAI on memory representation and retrieval, contrasting AI and neuroscience approaches. Current AI retrieval systems like RAG retrieve via fine-tuning and embedding similarity, while the presenter argued for exploring retrieval via combinatorial object identity or spatial proximity. The research explores circuit-level retrieval via domain fine-tuned LLMs and distributed memory for image retrieval using semantic similarity. Why it matters: The work suggests structured databases and retrieval-focused training can allow smaller models to outperform larger general-purpose models, offering efficiency gains for AI development in the region.
A KAUST alumnus presented research on using large language models for complex disease modeling and drug discovery. LLMs were trained on insurance claims of 123 million US people to model diseases and predict genetic parameters. Protein language models were developed to discover remote homologs and functional biomolecules, while RNA language models were used for RNA structure prediction and reverse design. Why it matters: This work highlights the potential of LLMs to accelerate computational biology research and drug development, with a KAUST connection.
Professor Won from KAIST presented a talk at MBZUAI on ensuring storage order in modern IO stacks. He discussed separating durability and ordering mechanisms to avoid expensive transfer-and-flush methods. The talk covered order-preserving IO stacks for single-queue block devices, multi-queue IO stacks, and all-flash arrays. Why it matters: Optimizing IO stacks is crucial for improving the performance and efficiency of storage systems in AI infrastructure and data centers.
A new neural network architecture called Orchid was introduced that uses adaptive convolutions to achieve quasilinear computational complexity O(N logN) for sequence modeling. Orchid adapts its convolution kernel dynamically based on the input sequence. Evaluations across language modeling and image classification show that Orchid outperforms attention-based architectures like BERT and Vision Transformers, often with smaller model sizes. Why it matters: Orchid extends the feasible sequence length beyond the practical limits of dense attention layers, representing progress toward more efficient and scalable deep learning models.
Joonhyuk Kang from KAIST gave a presentation at MBZUAI on AI's impact on wireless communication. The talk covered how communication systems can improve AI and how AI can develop wireless systems. Kang's research interests include signal processing for information transmission, security, and machine cognition. Why it matters: This talk highlights the growing intersection of AI and communication technologies in the region, with potential applications for smart cities and autonomous systems.
This article discusses the reliability of Deep Neural Networks (DNNs) and their hardware platforms, especially regarding soft errors caused by cosmic rays. It highlights that while DNNs are robust against bit flips, errors can still lead to miscalculations in AI accelerators. The talk, led by Prof. Masanori Hashimoto from Kyoto University, will cover identifying vulnerabilities in neural networks and reliability exploration of AI accelerators for edge computing. Why it matters: As DNNs are deployed in safety-critical applications in the region, ensuring the reliability of AI hardware is crucial for safe and trustworthy operation.
This article discusses the application of AI in semiconductor chip design and manufacturing, with a focus on examples such as IR-drop estimation and lithography processes. It mentions Youngsoo Shin, a KAIST professor and founder of Baum, who is an expert in this area. The article also briefly mentions panel discussion hosted by MBZUAI. Why it matters: AI-driven chip design and manufacturing could accelerate semiconductor innovation in the GCC region and beyond.
Dr. Zeke Xie from HKUST(GZ) presented research on noise initialization and sampling strategies for diffusion models. The talk covered golden noise for text-to-image models, zigzag diffusion sampling, smooth initializations for video diffusion, and leveraging image diffusion for video synthesis. Xie leads the xLeaF Lab, focusing on optimization, inference, and generative AI, with previous experience at Baidu Research. Why it matters: The work addresses core challenges in improving the quality and diversity of generated content from diffusion models, a key area of advancement for AI applications in the region.
Dr. Zhiqiang Lin from Ohio State University presented the Security-Enhanced Radio Access Network (SE-RAN) project to address cellular network threats using O-RAN. The project includes 5G-Spector, a framework for detecting L3 protocol exploits via MobiFlow and MobieXpert, and 5G-XSec, a framework leveraging deep learning and LLMs for threat analysis at the network edge. Dr. Lin also outlined a vision for AI convergence with cellular security for enhanced threat detection. Why it matters: Enhancing 5G security through AI and open architectures is critical for protecting next-generation mobile networks in the GCC region and globally.
Muhammad Shafique from NYU Abu Dhabi discusses building energy-efficient and robust EdgeAI systems. The talk covers trends, challenges, and techniques for optimizing software and hardware stacks. These optimizations aim to enable embodied AI in autonomous systems, IoT-Healthcare, Industrial-IoT, and smart environments. Why it matters: The research addresses key challenges in deploying AI on resource-constrained edge devices in the GCC region, particularly regarding energy efficiency and security.
Vicky Kalogeiton from École Polytechnique discussed the importance of multimodality for story-level recognition and generation using video, audio, text, masks and clinical data. She presented on multimodal video understanding using FunnyNet-W and Short Film Dataset. She further showed examples of visual generation from text and other modalities (ET, CAD, DynamicGuidance). Why it matters: Multimodal AI research is growing globally, and this talk highlights the potential of combining different data types for enhanced understanding and generation, which could have implications for various applications, including those relevant to the Middle East.
Researchers have developed robotic path-planning and control algorithms for minimally invasive surgery (MIS) that steer flexible needles, incorporating teleoperation and haptic feedback. An AI algorithm was designed to predict target motion due to respiratory movement, improving needle placement accuracy. GANs were used to generate synthetic images visualizing organ and tumor motion. Why it matters: This research demonstrates the potential of AI and robotics to enhance precision and adaptability in MIS, potentially reducing patient trauma and improving recovery times in the region and beyond.
The article discusses immersive analytics, which uses VR and AR to visualize data in 3D and embed it into the user's environment, and reviews systems and techniques from the Data Visualisation and Immersive Analytics lab at Monash University. It explores the concept of "embodied sensemaking" and its potential to improve how people work with complex data. Professor Tim Dwyer directs the Data Visualisation and Immersive Analytics Lab at Monash University. Why it matters: Immersive analytics could significantly enhance data comprehension and decision-making across various sectors in the Middle East, where large-scale projects and smart city initiatives generate vast datasets.
Pascal Fua from EPFL gave a talk at MBZUAI on physics-based deep learning for medical imaging. The talk covered how self-supervision and knowledge of human anatomy and physics can improve deep learning algorithms when training data is limited. Applications discussed included endoscopic heart surgery, colonoscopy, and intubation. Why it matters: This highlights the growing importance of domain knowledge and self-supervision in overcoming data scarcity challenges for AI in healthcare applications within the region.
A professor from Nanyang Technological University (NTU), Singapore gave a talk at MBZUAI about "Just-Noticeable Difference (JND)" models in visual intelligence. The talk covered visual JND models, research and applications, and future opportunities for JND modeling. JND can help tackle big data challenges with limited resources by focusing on user-centric and green systems. Why it matters: Exploring JND could lead to advancements in AI applications related to visual signal processing, image synthesis, and generative AI in the region.
Maha Elgarf from NYU Abu Dhabi presented research on using social robots to stimulate creativity in children through subconscious mimicry, leveraging the 'chameleon effect'. The research involved a series of studies where children engaged in storytelling with a social robot, and their creativity was assessed. Elgarf also discussed using Large Language Models (LLMs) in education and challenges in the field. Why it matters: This explores innovative applications of social robotics and AI in education within the UAE, potentially enhancing children's learning and creativity.
This research introduces a novel method using the Lateral Accretive Hybrid Network (LEARNet) to capture and analyze micro-expressions for mental health applications. The method refines both broad and subtle facial cues to detect mental health conditions like anxiety or depression. The authors also propose a neural architecture search (NAS) strategy to design a compact CNN for micro-expression recognition, improving performance and resource use. Why it matters: By integrating micro-emotion recognition with mental health estimation, the approach enables more accurate and early detection of emotional and mental health issues, potentially leading to improved well-being.
Video motion magnification amplifies subtle movements in video footage, making the imperceptible visible across various fields. In healthcare, it allows non-invasive monitoring of vital signs and micro-expressions. In engineering, it helps detect structural vibrations in infrastructure, while also being used in sports science, security, and robotics. Why it matters: The technology's ability to reveal hidden details has the potential to revolutionize diagnostics, monitoring, and decision-making in diverse sectors across the Middle East.
Marc Pollefeys from ETH Zurich and Microsoft Spatial AI Lab will discuss building 3D environment representations for assisting humans and robots. The talk covers visual 3D mapping, localization, spatial data access, and navigation using geometry and learning-based methods. It also explores building rich 3D semantic representations for scene interaction via open vocabulary queries leveraging foundation models. Why it matters: Advancements in spatial AI and 3D scene understanding are critical for enabling more capable robots and AI assistants in various applications within the region.
Gregory Chirikjian presented an overview of research on robot navigation in unstructured environments, using computer vision, sensor tech, ML, and motion planning. The methods use multi-modal observations from RGB cameras, 3D LiDAR, and robot odometry for scene perception, along with deep RL for planning. These methods have been integrated with wheeled, home, and legged robots and tested in crowded indoor scenes, home environments, and dense outdoor terrains. Why it matters: This research pushes the boundaries of robotics in complex environments, paving the way for more versatile and autonomous robots in the Middle East.
Jan Buchmann from TU Darmstadt presented research on NLP for long, structured documents at MBZUAI. The research addresses gaps in using document structure and improving the verifiability of LM responses. Experiments showed that models learn to represent document structure during pre-training, and larger models can cite sources well. Why it matters: This research contributes to making NLP more effective for complex documents like scientific articles and legal texts, which is crucial for information accessibility.
Michael Yu Wang, Chair Professor and Founding Dean of the School of Engineering at Great Bay University, argues for combining "good old fashioned engineering" (GOFE) with learning-based approaches like LLMs for robot skill acquisition, particularly in manipulation. He suggests a modular framework that integrates engineering principles with learning, drawing inspiration from human hand-eye coordination and tactile perception. Wang emphasizes the need to address engineering features of robot tactile sensors, such as spatial and temporal resolutions, to achieve human-like robot manipulation skills. Why it matters: This perspective highlights the importance of hybrid approaches combining traditional engineering with modern AI for advancing robotics, especially in complex manipulation tasks relevant to industries in the GCC region.
Professor Cesare Stefanini will discuss the relationship between AI and natural systems, highlighting robotics inspired by nature. His talk will cover neuro-inspired robot control, bio-hybrid ecosystems, and advancements in biomedical and industrial robotics. Stefanini will share achievements in underwater robot locomotion, AI-powered biomedical systems, and industrial platforms enhancing human manipulation. Why it matters: The talk at Khalifa University reflects the UAE's interest in bio-robotics research and integrating AI into various sectors, potentially fostering collaborations and advancements in the field.
A talk explores multimodal approaches inspired by user behavior for detecting deepfakes, considering user studies on multicultural deepfakes and the ACM Multimedia 2024 benchmark. The research leverages insights into how different audiences perceive manipulated media. Abhinav Dhall from Flinders University will present findings and future directions in deepfake analysis at MBZUAI. Why it matters: Addressing deepfakes is crucial for maintaining trust in digital content, especially with the increasing sophistication and accessibility of AI-driven manipulation tools.
Eliseo Ferrante from NYU Abu Dhabi presented work on increasing the controllability of swarm robotics systems. The research covers microscopic control via implicit intelligent leaders and macroscopic control via automated generation of swarm behaviors. Grammatical evolution and generative AI methods are used to produce collective behaviors aligned with human specifications. Why it matters: This research enhances the applicability of swarm robotics in real-world scenarios by improving control and coordination, potentially impacting industries like logistics, environmental monitoring, and disaster response in the region.
This article discusses a talk on "Assistive Augmentation," designing human-computer interfaces to augment human abilities. Examples include 'AiSee' for blind users, 'Prospero' for memory training, and 'MuSS-Bits' for deaf users to feel music. Suranga Nanayakkara from the National University of Singapore will present the talk, highlighting insights from psychology, human-centered machine learning, and design thinking. Why it matters: Such assistive technologies can significantly improve the quality of life for individuals with disabilities and extend human capabilities.
This article discusses the need for a decentralized approach to AI, especially in contexts where data and knowledge are distributed. It highlights five key technical challenges: privacy, verifiability, incentives, orchestration, and crowdUX. The author, Ramesh Raskar from MIT Media Lab, advocates for integrating privacy tech, distributed verifiable AI, data markets, orchestration, and crowd experience into the Web3 framework. Why it matters: Decentralized AI could unlock new possibilities for collaboration and problem-solving in the region, particularly in sectors like healthcare and logistics where data is often siloed.
Holger Pirk from Imperial College London is developing a novel approach to data management system composition called BOSS. The system uses a homoiconic representation of data and code and partial evaluation of queries by components, drawing inspiration from compiler-construction research. BOSS achieves a fully composable design that effectively combines different data models, hardware platforms, and processing engines, enabling features like GPU acceleration and generative data cleaning with minimal overhead. Why it matters: This research on composable database systems can broaden the applicability of data management techniques in the GCC region, enabling more flexible and efficient data processing for various applications.
Tetsunari Inamura's talk explores using VR to collect HRI data and tailor assistive robotic functionalities to individual users. He discusses symbol emergence via multimodal interaction, interactive behavior generation through symbol manipulation, and VR for data collection. The talk emphasizes long-term human capability enhancement and avoiding over-reliance on technology. Why it matters: This research promotes independence and growth in human-robot interactions, potentially revolutionizing assistive technologies in the region.
Hassan Sajjad from Dalhousie University presented research on exploring the latent space of AI models to assess their safety and trustworthiness. He discussed use cases where analyzing latent space helps understand the robustness-generalization tradeoff in adversarial training and evaluate language comprehension. Sajjad's work aims to build better AI models and increase trust in their capabilities by looking at model internals. Why it matters: Intrinsic evaluation of model internals will become important to improving AI safety and robustness.
Song Chaoyang from the Southern University of Science and Technology (SUSTech) presented research on Vision-Based Tactile Sensing (VBTS) for robot learning, combining soft robotic design with learning algorithms to achieve state-of-the-art performance in tactile perception. Their VBTS solution demonstrates robustness up to 1 million test cycles and enables multi-modal outputs from a single, vision-based input, facilitating applications such as amphibious tactile grasping and industrial welding. The talk also highlighted the DeepClaw system for capturing human demonstration actions, aiming for a universal interaction interface. Why it matters: This research advances embodied intelligence by improving robot dexterity and adaptability through enhanced tactile sensing, which is crucial for complex manipulation tasks in various sectors such as manufacturing and healthcare within the region.
Mykel Kochenderfer from Stanford University gave a talk on building robust decision-making systems for autonomous systems, highlighting the challenges of balancing safety and efficiency in uncertain environments. The talk addressed computational tractability and establishing trust in these systems. Kochenderfer outlined methodologies and research applications for building safer systems, drawing from his work on air traffic control, unmanned aircraft, and automated driving. Why it matters: The development of safe and reliable autonomous systems is crucial for various applications in the region, and insights from experts like Kochenderfer can guide research and development efforts at institutions like MBZUAI.
Dr. Hao Dong from Peking University presented research on addressing the challenge of limited large-scale training data in embodied AI, particularly for manipulation, task planning, and navigation. The presentation covered simulation learning and large models. Dr. Dong is a chief scientist of China's National Key Research and Development Program and an area chair/associate editor for NeurIPS, CVPR, AAAI, and ICRA. Why it matters: Overcoming data scarcity is crucial for advancing embodied AI research and enabling more sophisticated robotic applications in the region.
Krishna Murthy, a postdoc at MIT, researches computational world models to enable robots to understand and operate effectively in the physical world. His work focuses on differentiable computing approaches for spatial perception and interfaces large image, language, and audio models with 3D scenes. Murthy envisions structured world models working with scaling-based approaches to create versatile robot perception and planning algorithms. Why it matters: This research could significantly advance robotics by enabling more sophisticated perception, reasoning, and action capabilities in embodied agents.
This seminar explores vision systems through self-supervised representation learning, addressing challenges and solutions in mainstream vision self-supervised learning methods. It discusses developing versatile representations across modalities, tasks, and architectures to propel the evolution of the vision foundation model. Tong Zhang from EPFL, with a background from Beihang University, New York University, and Australian National University, will lead the talk. Why it matters: Advancing vision foundation models is crucial for expanding AI applications, especially in the Middle East where computer vision can address challenges in areas like urban planning, agriculture, and environmental monitoring.
Pedro J. Moreno, former head of ASR R&D at Google, presented a talk at MBZUAI on the past, present, and future of speech technologies. The talk covered the evolution of speech tech, his career contributions including work on Google Voice search, and the impact of LLMs on speech science. He also discussed the interplay between foundational and applied research and preparing the next generation of scientists. Why it matters: The talk provides insights into the trajectory of speech technologies from a leading researcher, highlighting future directions and the ethical considerations surrounding AI's impact on society.
Eduardo da Veiga Beltrame, bioinformatics lead at ImYoo (a Caltech spinout), presented on scalable methods for single-cell omics data analysis, including kallisto|bustools and scvi-tools. He highlighted their use in ImYoo's decentralized longitudinal study on Inflammatory Bowel Disease (IBD), where patients self-collect capillary blood samples. Beltrame also discussed his research on STEM education programs in Brazil as a visiting scholar at UC Berkeley. Why it matters: This highlights the growing trend of decentralized clinical studies leveraging advanced single-cell technologies for precision medicine, showcasing the potential of remote data collection and analysis in understanding complex diseases.
Amir Goharshady from Hong Kong University of Science and Technology presented a talk at MBZUAI on martingale-based verification of probabilistic programs. The talk covered using martingale-based approaches for proving termination and synthesizing cost bounds for probabilistic programs, automating program analysis with template-based methods. He also discussed remaining challenges and open problems in the area. Why it matters: Advances in formal verification and analysis of probabilistic programs are crucial for ensuring the reliability and safety of AI systems that rely on randomization.
Kimon Fountoulakis from the University of Waterloo presented a talk on machine learning on graphs, covering node classification and algorithmic reasoning. The talk discussed the limitations and strengths of graph neural networks (GNNs). It also covered novel optimal architectures for node classification and the ability of looped GNNs to execute classical algorithms. Why it matters: Understanding GNN capabilities is crucial for advancing AI applications in areas like recommendation systems and drug discovery that rely on relational data.
NYU Abu Dhabi hosted a talk by Prof. Debdeep Mukhopadhyay on the intersection of machine learning and hardware security. The talk covered using ML/DL for side-channel attacks, leakage assessment in crypto-devices, and threats to hardware security primitives. Prof. Mukhopadhyay is a visiting professor at NYU Abu Dhabi and Institute Chair Professor at IIT Kharagpur. Why it matters: The talk highlights the growing importance of hardware security in modern systems and the role of machine learning in both attacking and defending hardware vulnerabilities.
A senior lecturer at the University of New South Wales discussed the use of AI to improve early prognosis and personalized treatment plans for neurodegenerative diseases, cardiovascular imaging and multiomics. The lecture highlighted the potential of AI algorithms to detect subtle changes at early stages through advanced multiomics techniques and medical imaging analysis. The speaker has expertise in analyzing medical images and has collaborated with medical professionals to develop AI tools for diagnosis of cancer, neurodegenerative disease, and heart disease. Why it matters: AI-driven prognosis and treatment planning promises earlier intervention and improved outcomes for challenging diseases in the region.
MBZUAI's Dr. Artem Shelmanov is working on uncertainty quantification (UQ) methods for generative LLMs to detect unreliable generations. He aims to address the issue of LLMs fabricating facts, often called "hallucinating", without clear indicators of veracity. He systemizes existing UQ efforts, discusses caveats, and suggests novel techniques for safer LLM use. Why it matters: Improving the reliability of LLMs is crucial for responsible AI deployment in the region, especially in sensitive applications.
Dr. Jindong Wang from Microsoft Research Asia gave a talk at MBZUAI about the limitations of large foundation models, including adapting to real-world unpredictability and security concerns. He also discussed the need for interdisciplinary collaboration to evaluate the benefits and risks of these models. Dr. Wang shared his research and insights on how to harness the power of large foundation models while addressing their constraints and fostering responsible AI integration. Why it matters: This highlights MBZUAI's role in hosting discussions about responsible AI development and the challenges of deploying foundation models.
Ekaterina Radionova from Smarter AI (formerly Samsung AI Center) presented an approach to generating lifelike real-time avatars. The work focuses on generating high-quality video with authentic facial features to support online generation. Radionova's master's degree is from Skoltech on Data Science program and Bachelor degree at Moscow Institute of Physics and Technology on Applied Math. Why it matters: Achieving realistic real-time avatars is critical for applications in online communication, entertainment, and virtual reality within the region.
Giuseppe Loianno from NYU presented research on creating "Super Autonomous" robots (USARC) that are Unmanned, Small, Agile, Resilient, and Collaborative. The research focuses on learning models, control, and navigation policies for single and collaborative robots operating in challenging environments. The talk highlighted the potential of these robots in logistics, reconnaissance, and other time-sensitive tasks. Why it matters: This points to growing research interest in advanced robotics in the region, especially given the focus on smart cities and automation.