Multi-Fidelity Methods: State of the Art and Open Challenges
Multi-fidelity (MF) methods exploit correlations between low- and high-fidelity models to reduce the number of expensive evaluations required for prediction and optimization. A classical approach is co-Kriging (Kennedy-O'Hagan), which models the high-fidelity response through an autoregressive Gaussian process (GP) relationship with the low-fidelity response. Extensions include nonlinear information fusion with GPs, Bayesian multi-fidelity inference and deep probabilistic surrogates, as well as MF neural networks that learn nonlinear cross-fidelity correlations.
More broadly, scientific machine learning methods such as physics-informed neural networks (PINNs) and operator learning (DeepONet, Fourier Neural Operator) provide scalable tools to learn mappings between function spaces, which is particularly relevant when model outputs are fields and discretizations differ.
Despite substantial progress, a key limitation remains insufficiently resolved: heterogeneous inputs and outputs across fidelities. In many applications, the low- and high-fidelity models do not share the same parameterization, discretization, or state variables. Such heterogeneity prevents the direct application of standard MF frameworks that assume a shared input space and pointwise correspondence between outputs.
PhD Objectives
The objective is to design scalable multi-fidelity methods capable of merging information coming from models with mismatched parameterizations and heterogeneous data structures.
The PhD will investigate i) how to define common latent representations linking heterogeneous input spaces across fidelities; ii) how to build multi-fidelity surrogates that remain consistent under such heterogeneities; iii) how to quantify and propagate uncertainty induced by limited high-fidelity data and representation mismatch, iv) how to design adaptive sampling strategies for selecting expensive high-fidelity evaluations.
Scientific Methodology and Work Plan
The project is structured around four main axes.
The first axis is around the building of controlled benchmark settings where fidelity levels differ in parameterizations and/or discretizations (e.g., reduced vs.\ full-order models, coarse vs.\ fine meshes, sparse sensors vs.\ full fields). The goal is to formalize sources of heterogeneity (input mismatch, output mismatch, partial observability, missing variables) and define the problem mathematically.
Secondly, a central component of the PhD will be to learn mappings between heterogeneous spaces through latent-variable models and representation learning. Some methods that will be explored rely on the following techniques: i) Autoencoders / Variational Autoencoders (VAE) to embed high-dimensional inputs (fields, images, mesh-based signals) into low-dimensional latent coordinates; ii) Cross-modal alignment (e.g., canonical correlation analysis and Deep CCA) to align heterogeneous parameterizations and modalities in a shared latent space, iii) Operator learning (DeepONet, FNO) to learn mappings between function spaces and provide a bridge between heterogeneous inputs and field outputs.
The emphasis will be on representations that preserve physical meaning, support generalization, and remain compatible with downstream MF inference and uncertainty quantification.
Building on learned representations, the PhD will develop MF surrogate models that fuse low- and high-fidelity information. In particular, some techniques will be explored and compared:
- Latent-space co-Kriging: define GP models in a shared latent space and propagate uncertainty induced by the embedding.
- Multi-fidelity neural surrogates: residual learning and hierarchical neural architectures conditioned on fidelity indicators and latent variables.
- Hybrid probabilistic-deep models: combine neural representations with probabilistic heads (GPs, Bayesian neural networks) for calibrated uncertainty estimates.
Finally, the PhD candidate will focus on \textbf{active learning / adaptive design} for MF settings with heterogeneous inputs. The goal is to decide where to run expensive high-fidelity simulations to maximize information gain.
[1] M. C. Kennedy and A. O’Hagan, Predicting the output from a complex computer code when fast approximations are available, Biometrika, 87(1), 1–13 (2000).
[2] P. Perdikaris, M. Raissi, and G. E. Karniadakis, Nonlinear information fusion algorithms for data-efficient multi-fidelity modeling, Proc. Roy. Soc. A, 473(2198):20160751 (2017).
[3] B. Peherstorfer, K. Willcox, and M. Gunzburger, Survey of multifidelity methods in uncertainty propagation, inference, and optimization, SIAM Review, 60(3):550–591 (2018).
[4] Diederik P Kingma and Max Welling, Auto-Encoding Variational Bayes, International Conference on Learning Representations (ICLR) 2014 ArXiv. http://arxiv.org/abs/1312.6114.
[5] X. Meng and G. E. Karniadakis, A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems, JCP, 401, 109020 (2020).
[6] M. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear PDEs, J. Comput. Phys., 378:686–707 (2019).
[7] Y. Yang, P. Perdikaris, Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems, arXiv:1901.04878 (2019).
8] L. Lu, J. Pengzhan, P. Guofei, G. Karniadakis, Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators, Nature Machine Intelligence, 3-3, 218–229, (2021).
[9] Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, Fourier Neural Operator for Parametric Partial Differential Equations, arXiv:2010.08895, 2020.
[10] B. Shahriari, K. Swersky, Z. Wang, R. P. Adams and N. de Freitas, Taking the Human Out of the Loop: A Review of Bayesian Optimization, Proceedings of the IEEE, 104-1, 148-175, 2016.
