A talk introduces a computational framework for learning a compact structured representation for real-world datasets, that is both discriminative and generative. It proposes to learn a closed-loop transcription between the distribution of a high-dimensional multi-class dataset and an arrangement of multiple independent subspaces, known as a linear discriminative representation (LDR). The optimality of the closed-loop transcription can be characterized in closed-form by an information-theoretic measure known as the rate reduction. Why it matters: The framework unifies concepts and benefits of auto-encoding and GAN and generalizes them to the settings of learning a both discriminative and generative representation for multi-class visual data.
KAUST Ph.D. student Jinhui Xiong won the best paper award at the 24th International Symposium on Vision, Modeling, and Visualization in Germany for his paper "Stochastic Convolutional Sparse Coding". The paper, co-authored with KAUST Professors Peter Richtárik and Wolfgang Heidrich, introduces a novel stochastic spatial-domain solver for Convolutional Sparse Coding (CSC). The proposed algorithm outperforms state-of-the-art solutions in terms of execution time and offers an improved representation for learning dictionaries from sample images. Why it matters: This award recognizes significant research in efficient image representation and dictionary learning, contributing to advancements in visual computing and AI at KAUST.
A new neural network architecture called Orchid was introduced that uses adaptive convolutions to achieve quasilinear computational complexity O(N logN) for sequence modeling. Orchid adapts its convolution kernel dynamically based on the input sequence. Evaluations across language modeling and image classification show that Orchid outperforms attention-based architectures like BERT and Vision Transformers, often with smaller model sizes. Why it matters: Orchid extends the feasible sequence length beyond the practical limits of dense attention layers, representing progress toward more efficient and scalable deep learning models.