Scalable Gaussian Processes

Staff - Faculty of Informatics

Date: 1 July 2022 / 14:00 - 16:30

USI East Campus, Room C1.03 & MS Teams

You are cordially invited to attend the PhD Dissertation Defence of Manuel Schuerch on Friday 1 July 2022 at 14:00 in room C1.03 (East Campus) and on MS Teams.

This thesis provides novel contributions to scalable Gaussian processes (GPs), which constitute an important tool in machine learning and statistics with applications ranging from social and natural science through engineering. GPs are powerful probabilistic methods with many benefits, such as their modeling flexibility, the robustness to overfitting and the availability of well-calibrated predictive uncertainty estimates. However, off-the-shelf GP inference procedures are limited to datasets with several thousand training data points because of their cubic computational complexity. This thesis presents new methodologies and novel algorithms for scaling GP regression to larger datasets by employing sequential and local methods. In particular, the first contribution of this work is a unifying GP approximation method based on a recursive formulation, which enables to train analytically a range of existing GP models in an online and distributed way. In this formulation, the so-called hyperparameters, which refer to a few parameters determining the GP, are assumed to be known. On the other hand, those can be learned by the second major contribution consisting of two novel algorithms for sequential hyperparameters estimation. These allow to scale the training of GPs up to millions of training samples. The last contribution involves a novel unifying GP approximation model exploiting sparsity and locality. Specifically, a method based on local GPs, which can share common information with a flexible correlation structure, is proposed. Thereby, this new model unifies several existing local and global GP approximation approaches. All the proposed methods in this thesis are theoretically supported and empirically tested on synthetic as well as real-world datasets with up to millions of training samples. Thereby, these new methods outperform the state-of-the-art in several tasks. This demonstrates the effectiveness of the novel GP approximations proposed in this thesis, which can achieve high-scalability without sacrifying the performance of original GPs. Therefore, this work substantially contributes to overcome the computational complexity barrier for the large-scale adoption of GPs.

Dissertation Committee:
- Prof. Luca Maria Gambardella, Università della Svizzera italiana, Switzerland (Research Advisor)
- Prof. Marco Zaffalon, IDSIA USI-SUPSI, Switzerland (Research co-Advisor)
- Prof. Alessio Benavoli, Trinity College Dublin, Ireland (Research co-Advisor)
- Prof. Cesare Alippi, Università della Svizzera italiana, Switzerland (Internal Member)
- Prof. Rolf Krause, Università della Svizzera italiana, Switzerland  (Internal Member)
- Prof. David Ginsbourger, University of Bern, Switzerland (External Member)
- Prof. Gianluigi Pillonetto, University of Padova, Italy (External Member)