Within the framework of a partnership (you can choose between)
- Collaboration with Kateek Alahari and David Salinas.
Is regular travel foreseen for this post ? Yes, travel costs will be covered within the limits of the scale in force.
Context and Objective:
Tabular and Time-series foundational models have become very popular due to their high accuracy relying solely on In-Context Learning (ICL) [3, 4]. However, one still requires training one model per ”modality”, e.g., one model for regression, one model for classification (although recent work showed that it is possible to reuse the TabPFN regression checkpoint for time-series [5]).
One reason for this limitation is that models use a parametric projection to obtain predictions. This parametric projection has intrinsic limitations, forcing one to learn a model that can predict only up to the maximum dimension seen during training; it also requires padding. Two exceptions are recent works which instead propose to learn a model with an equivariant architecture, with either a non-parametric approach [6] or an equivariant architecture [1].
The project will build on this work and develop a single foundational model able to do high-dimensional classification and regression. The project will leverage previous work using non-parametric approaches to search for an efficient architecture and aim at having a single model able to perform well on TabArena [2]. If time permits, the student will look into multivariate extensions, e.g., predicting joint distributions instead of marginals as done in time-series forecasting with equivariant parametrization [8] or diffusion processes [7]. The student will also look into applications in tabular and time-series predictions.
