Tutorials
Tutorials can be attended by participants registering either in full registration or in student registration, by paying the corresponding fee of 25 Euros per tutorial.
Program: T1 – T4 Morning tutorials. |. T5 – T8 Afternoon tutorials
Program: T1 – T4 Morning tutorials
T1 - Title: Nonconvex Optimization for High-Dimensional Signal Estimation: Spectral and Iterative Methods
Presenters: Yuejie Chi (Carnegie Mellon University), Yuxin Chen (Princeton University), Cong Ma (Princeton University)
Abstract: Recent years have seen tremendous progress in the development of provable nonconvex information processing, with the aid of proper statistical models of data. Examples include recommendation systems, structured matrix recovery, phase retrieval, community detection, ranking, amongst others. The focal point of this tutorial is two approaches that have received much attention recently. (1) Spectral methods: which refer to a collection of algorithms built upon eigenanalysis of some properly arranged data matrices. While the studies of spectral methods can be traced back to classical matrix perturbation theory, the past decade has witnessed a tremendous theoretical advance with the aid of statistical modeling, particularly in the development of entrywise perturbation bounds that are much less known to the signal processing community. (2) Nonconvex iterative methods, which attempt to solve the nonconvex problems of interest via iterative optimization algorithms like gradient descent. Despite a high degree of nonconvexity, it has been shown that the loss surface of many high-dimensional signal processing problems may possess benign geometric structures either locally or globally, which in turn allow one to find optimal solutions efficiently. As a result, for a broad range of low-rank matrix estimation problems, nonconvex iterative methods are capable of achieving optimal statistical accuracy and computational efficiency simultaneously.
This tutorial will focus on both spectral methods and iterative optimization algorithms for estimating low-complexity models in high dimension, highlighting the following aspects:
- The role of statistical models: with the help of statistical models, one is allowed to leverage the population risk / loss functions that might be easier to characterize in terms of optimization geometry. The main challenge lies in how the optimization geometry translates back to the empirical loss. The tutorial will discuss common techniques and tools for developing such an analysis framework in both global (where a random initialization suffices) and local (where a careful initialization close to the desired solution is needed) approaches for optimization.
- The entrywise performance bounds: the tutorial will emphasize an entrywise eigenvector (or singular vector) perturbation analysis, a fundamentally important topic that has only recently become tractable thanks to the advances of statistical tools. Such performance bounds, which are previously rarely available, are of great interest in applications such as ranking, community detection, and localization.
- Case studies: the tutorial will feature several key applications and analysis techniques, and highlight their broad applicability.
T2 - Title: Statistical Relational Artificial Intelligence for Unified Music Understanding and Creation
Presenters: Carmine-Emanuele Cella (Center for New Music and Audio Technology (CNMAT), University of California, Berkeley, Helene-Camille Crayencour (Centre National de la Recherche Scientifique), George Tzanetakis (Department of Computer Science, University of Victoria, Canada)
Abstract: Understanding music audio signals requires dealing with uncertainty and complex relational structure at multiple levels of representation and integrating information from multiple tasks. Traditionally these issues have been treated separately. Uncertainty in knowledge has been addressed with probability and learning techniques while knowledge and complex relational information has been modeled using logic representations. Traditional approaches for music processing are not able to cope with its rich, highly structured, relational structure. Logic can handle the complexity of the real world and is capable of reasoning with large numbers of interacting heterogeneous objects, but it cannot deal with its uncertainty. Probabilistic graphical models are a powerful framework for dealing with uncertainty, but they can’t handle real- world complexity. Deep learning is able to create complex abstract representations from large-scale raw data but its mechanism for structure learning remains to be understood. In this tutorial we present recent development in the area of Statistical Relational Artificial Intelligence (StarAI) that unifies probability, logic and (deep) learning. We describe how existing approaches to music information retrieval (MIR) can find powerful extensions and unification in StarAI and outline future opportunities and challenges in future research possibilities.
T3 - Title: Advances in Massive MIMO Hybrid Beamforming: From Optimization to Learning
Presenters: Kumar Vijay Mishra (U. S. Army Research Laboratory) and Christos G. Tsinos (University of Luxembourg)
Abstract: Next-generation millimeter-wave (mm-Wave) communications offer wider bandwidth and higher data rates than conventional cellular systems. Hybrid beamforming techniques are key to successfully realizing the massive multiple-input-multiple-output (MIMO) antenna architectures for mm-Wave communications. The design of the hybrid beamformers requires solving difficult nonconvex optimization problems that involve a common performance metric, i.e. spectral efficiency or energy efficiency as a cost function and a number of constraints related to the employed communication regime and the adopted architecture of the hybrid system. Typical architectures involve a low dimensional baseband counterpart connected to the antenna array via an analog phase shifting network. Thus, the hybrid beamformer admits a factorized form constituted by a beamforming matrix for the BB counterpart and a matrix with unit-modulus entries, i.e. the phase shifts of the analog network. There is no standard methodology for solving such problems and usually, the derivation of an efficient solution is a very challenging task. Various optimization-based approaches have been suggested that are tuned to specific scenarios and requirements. However, these methods suffer from high computational complexity and their performance strongly relies on the perfect channel condition. Some of these challenges are offset by employing deep learning (DL) techniques which offer advantages such as low computational complexity while solving optimization-based or combinatorial search problems in hybrid beamforming. In this tutorial, the audience will learn ways to design hybrid beamforming transceivers by decoupling the transmitter(s)-receiver(s) problem and applying different optimization frameworks for deriving efficient algorithmic solutions. The different approaches will be thoroughly analyzed and compared with respect to their performance and computational complexity. The application of the proposed methodology to the design of hybrid beamformers for single-user MIMO, multi-user MIMO, cognitive radio, cooperative frequency-selective systems will also be discussed. Finally, the audience will learn about applying DL to various aspects of hybrid beamforming including channel estimation, antenna selection, the effect of quantization, wideband beamforming for frequency-selective channels, knowledge transfer across various geometries, and spatial modulation.
T4 - Title: A Unified Framework for Underdetermined and Determined Blind Audio Source Separation
Presenters: Nobutaka Ito (NTT Corporation, Japan) and Hiroshi Sawada (NTT Corporation, Japan)
Abstract: This tutorial presents a unified framework for underdetermined and determined Blind Source Separation (BSS) for audio signals. The framework encompasses eight BSS methods as particular examples, namely complex-Gaussian/second-order-statistics Independent Component Analysis (ICA), Rank-1 spatial Covariance Analysis (R1CA), Full-rank spatial Covariance Analysis (FCA), FastFCA, Independent Low-Rank Matrix Analysis (ILRMA), rank-1 Multichannel Nonnegative Matrix Factorization (MNMF), full-rank MNMF, and FastMNMF. The framework enables us to garner a unified understanding of the BSS methods. Especially, it elucidates close relation between underdetermined and determined BSS that have been studied somewhat independently of each other, hopefully creating synergy between these research areas and stimulating further advances of them. The tutorial features a live BSS demo, where three volunteers are rounded up from the audience and asked to read a sentence simultaneously. A recording will be made with an IC recorder equipped with two microphones, and separated into individual speakers' speech using some BSS methods explained in the tutorial.
Program: T5 – T7 Afternoon tutorials
T5 - Title: Four decades of array signal processing research: an optimization relaxation technique perspective
Presenters: Marius Pesavento (TU Darmstadt), Minh Trinh-Hoang (TU Darmstadt), Mats Viberg (Blekinge Institute of Technology)
Abstract: In our community, we currently witness that sensor array processing, more specifically direction-of- arrival (DoA) estimation, receives new momentum due to the emergence of new applications such as automotive radar, drone localization, parametric channel estimation in Massive MIMO. This development is further inspired by the emergence of new powerful and affordable multiantenna hardware platforms. Today, we are looking back at a history of more than four decades of super resolution DoA estimation techniques, starting from the early parametric methods such as the computationally expensive maximum likelihood (ML) techniques to the early subspace based techniques such as Pisarenko, MUSIC, ESPRIT, and MODE.
In this tutorial, we provide a consistent overview of developments in four decades of sensor array processing techniques for DoA estimation, ranging from traditional super resolution techniques to modern sparse optimization based DoA estimation techniques. In our overview, we take a modern optimization based view and retell the story of sensor array processing from the relaxation technique perspective. We will show, from our perspective, that constrained optimization problem formulations and problem relaxation techniques have always played an important role in the development of powerful DoA estimation methods. This optimization relaxation viewpoint will provide novel insight into the design and analysis of existing algorithms. Furthermore, as a consequent step forward along the line of relaxation techniques in DoA estimation, we will introduce the partial relaxation framework. The basic concept of the partial relaxation framework is intuitive and simple, and it can be applied to various multisource DoA estimation criteria. In contrast to classical multisource estimation methods where costly optimization is carried out over the multi-dimensional array manifold, in partial relaxation the manifold structure is maintained only for a single steering vector while the manifold structure is relaxed for the remaining steering vectors. This relaxation, despite still nonconvex, often admits closed form solutions for the nuisance parameters, which in turn results in computationally tractable algorithms. The estimators constructed under the partial relaxation framework show remarkable performance in the practically important threshold domain, i.e., in scenarios where the sources are poorly separated, the SNR is low or the sample number is small.
T6 - Title: Machine Learning and Wireless Communications
Presenters: Nir Shlezinger (Weizmann Institute), Yonina C. Eldar (Weizmann Institute), H. Vincent Poor (Princeton University)
Abstract: Mobile communications and machine learning are two of the most exciting and rapidly developing technological fields of our time. In the past few years, these two fields have begun to merge in two fundamental ways. First, while mobile communications has developed largely as a model-driven field, the complexities of many emerging communication scenarios raise the need to introduce data-driven methods into the design and analysis of mobile networks. Second, many machine learning problems are by their nature distributed due to either physical limitations or privacy concerns. This distributed nature can be exploited by using mobile networks as part of the learning mechanisms, i.e., as platforms for machine learning.
In this tutorial we will illuminate these two perspectives, presenting a representative set of relevant problems which have been addressed in the recent literature, and discussing the multitude of exciting research directions which arise from the combination of machine learning and wireless communications. We will begin with the application of machine learning methods for optimizing wireless networks: Here, we will first survey some of the challenges in communication networks which can be treated using machine learning tools. Then, we will focus on one of the fundamental problems in digital communications – receiver design. We will review different designs of data-driven receivers, and discuss how they can be related to conventional and emerging approaches for combining machine learning and model-based algorithms. We will conclude this part of the tutorial with a set of communication-related problems which can be tackled in a data-driven manner.
The second part of the tutorial will be dedicated to wireless networks as a platform for machine learning: We will discuss communication issues arising in distributed learning problems such as federated learning and collaborative learning. We will explain how established communications and coding methods can contribute to the development of these emerging distributed learning technologies, illustrating these ideas through examples from recent research in the field. We will conclude with a set of open machine learning related problems, which we believe can be tackled using established communications and signal processing techniques.
T7 - Title: Adaptive Optimization Methods for Machine Learning and Signal Processing
Presenters: Kfir Y. Levy (Technion), Ahmet Alacaoglu (EPFL), Ali Kavis (EPFL) Volkan Cevher (EPFL)
Abstract: In this tutorial, we are going to focus on several aspects of adaptivity for optimization problems ranging from function minimization to min-max optimization, covering applications such as neural network and generative adversarial network (GAN) training, reinforcement learning, sparse recovery and inverse problems. Our exposition will shed light onto the underlying mechanisms and analyses of the algorithms, such as online learning and regret minimization; and the know-how that is needed to make the most of them in applications. We will also illustrate the benefits of adaptive methods over their non-adaptive variants.