Exploring high-dimensional random landscapes: the case of multi-spiked tensor estimation

Speaker: Vanessa Piccolo, EPFL

Abstract: Modern machine learning algorithms rely on the optimization of highly nonconvex functions in very high dimensions. The associated loss landscapes typically exhibit an exponential number of stationary points, yet simple gradient-based methods such as Stochastic Gradient Descent (SGD) perform remarkably well in practice. Despite extensive empirical evidence, the theoretical mechanisms underlying this success remain poorly understood.
In many high-dimensional learning problems, the optimization landscape consists of a high-dimensional random component perturbed by a low-dimensional signal. A recurring phenomenon is that, despite the complexity of such landscapes, the dynamics of SGD can often be described in terms of a low-dimensional set of summary statistics capturing alignment with the underlying signal structure. Motivated by this, I will discuss an exactly solvable model: the canonical multi-spiked tensor estimation problem. In this setting, the goal is to recover multiple signal vectors on the high-dimensional sphere from noisy Gaussian tensor observations. I will present recent results on both geometric aspects of the associated loss landscape and the dynamical behavior of SGD as it explores this landscape. This talk is partially based on joint work with Gérard Ben Arous and Cédric Gerbelot.

Biography: Vanessa Piccolo is a postdoctoral researcher at EPFL in the Information, Learning and Physics laboratory, working with Florent Krzakala. Her work lies at the interface of probability theory and the mathematical foundations of data science and machine learning, with a particular emphasis on random matrix theory and high-dimensional probability. She obtained her Ph.D. in mathematics from ENS Lyon (France) under the co-supervision of Alice Guionnet (CNRS and ENS Lyon) and Gérard Ben Arous (NYU, Courant Institute). She previously studied mathematics at ETH Zurich.

Host: Prof. Ernst Wit