Draft:Simulation-based Inference

Simulation-based inference (SBI), also known as likelihood-free inference, is a family of statistical methods for performing Bayesian inference when a model of a system can be simulated but its likelihood function is unavailable, intractable, or too computationally expensive to evaluate directly. SBI addresses a central question in computational science: given a simulation model of a target system, what can be learned about unknown parameters from observed data? The same question arises in multiple disciplines, including Bayesian statistics, inverse problems, uncertainty quantification, and machine learning.

SBI methods use simulations to approximate posterior distributions over unknown parameters conditioned on observations. SBI encompasses methods such as approximate Bayesian computation (ABC), and more recent approaches based on neural density estimation and modern generative modeling.

Overview

In Bayesian inference, the objective is to infer unknown parameters ${\boldsymbol {\theta }}$ from an observation $\mathbf {y} _{\mathrm {obs} }$ using Bayes' theorem: $p({\boldsymbol {\theta }}\mid \mathbf {y} _{\mathrm {obs} })\propto p(\mathbf {y} _{\mathrm {obs} }\mid {\boldsymbol {\theta }})\,p({\boldsymbol {\theta }}),$ where $p({\boldsymbol {\theta }})$ is a prior distribution and $p(\mathbf {y} _{\mathrm {obs} }\mid {\boldsymbol {\theta }})$ is the likelihood.

In many scientific models, the likelihood cannot be evaluated even though it is possible to generate synthetic observations $\mathbf {y}$ from parameters ${\boldsymbol {\theta }}$: $\mathbf {y} \sim p(\mathbf {y} \mid {\boldsymbol {\theta }}).$ SBI procedures perform inference by using simulated pairs $({\boldsymbol {\theta }},\mathbf {y} )$ as a computational surrogate for likelihood evaluations.

Problem formulation

A standard SBI setup assumes an observation $\mathbf {y} _{\mathrm {obs} }\in {\mathcal {Y}}$, which may be high-dimensional and of arbitrary structure (e.g., vectors, sets, graphs, sequences, or mixed discrete–continuous data). The model defines a joint distribution $p({\boldsymbol {\theta }},\mathbf {y} )=p(\mathbf {y} \mid {\boldsymbol {\theta }})\,p({\boldsymbol {\theta }}),$ where $p({\boldsymbol {\theta }})$ specifies plausible parameters and $p(\mathbf {y} \mid {\boldsymbol {\theta }})$ describes how data are generated.

In likelihood-based models, the likelihood $p(\mathbf {y} \mid {\boldsymbol {\theta }})$ can be evaluated for any pair $({\boldsymbol {\theta }},\mathbf {y} )$. In simulation-based models, the data model is available only implicitly through a simulator that can generate synthetic observations but cannot compute likelihood values. SBI methods aim to infer parameters while providing calibrated uncertainty quantification, typically through an approximation of the posterior distribution $p({\boldsymbol {\theta }}\mid \mathbf {y} _{\mathrm {obs} })$.

Many SBI methods can be described in terms of three essential components:

1. A simulator that generates synthetic observations $\mathbf {y}$ given latent parameters ${\boldsymbol {\theta }}$.
2. A prior distribution $p({\boldsymbol {\theta }})$ that defines the domain of plausible parameters.
3. An approximator that uses simulations to infer ${\boldsymbol {\theta }}$ from observations $\mathbf {y}$, or to approximate posterior or likelihood-related quantities.

In modern SBI, the approximator is often a machine learning model (e.g., a neural network) trained on simulated parameter–observation pairs.

Methods

Simulation-based inference includes both classical likelihood-free algorithms and machine learning-based approaches.

Approximate Bayesian computation

Approximate Bayesian computation (ABC) is a classical family of likelihood-free methods that approximates the posterior by comparing observed and simulated data, often through summary statistics. In its rejection-sampling form, ABC samples ${\boldsymbol {\theta }}$ from the prior, simulates $\mathbf {y}$ from the model, and accepts ${\boldsymbol {\theta }}$ if the simulated data are sufficiently close to the observed data under a discrepancy threshold: $d(s(\mathbf {y} ),s(\mathbf {y} _{\mathrm {obs} }))<\varepsilon ,$ where $s(\cdot )$ denotes summary statistics, $d$ is a distance function, and $\varepsilon$ is a tolerance parameter. Variants include ABC-MCMC and ABC-SMC.

Neural simulation-based inference

Many modern SBI approaches learn flexible approximations using neural networks trained on simulated pairs $({\boldsymbol {\theta }},\mathbf {y} )$. A widely used perspective treats simulations as training data for conditional generative modeling, enabling inference in high-dimensional settings.

Common categories include:

• Neural posterior estimation (NPE), which trains a conditional density estimator $q_{\phi }({\boldsymbol {\theta }}\mid \mathbf {y} )$ to approximate the posterior $p({\boldsymbol {\theta }}\mid \mathbf {y} )$ directly. The trained model can be sampled immediately to obtain posterior samples without additional simulation or sampling algorithms. The approximation quality depends on coverage of the observation space during training, which can be challenging when observations are high-dimensional.
• Neural likelihood estimation (NLE), which trains a model $q_{\phi }(\mathbf {y} \mid {\boldsymbol {\theta }})$ to approximate the likelihood $p(\mathbf {y} \mid {\boldsymbol {\theta }})$, enabling the use of standard Bayesian sampling algorithms such as Markov chain Monte Carlo to draw posterior samples. NLE provides flexibility to change the prior distribution after training without retraining the network, since the learned likelihood approximation remains valid. However, NLE requires running MCMC at inference time.
• Neural ratio estimation (NRE), which estimates the likelihood-to-evidence ratio $r({\boldsymbol {\theta }},\mathbf {y} )={\frac {p(\mathbf {y} \mid {\boldsymbol {\theta }})p({\boldsymbol {\theta }})}{p(\mathbf {y} )}}$ to support posterior inference without explicit likelihood evaluation. NRE methods train classifiers to distinguish between samples from the joint distribution $p({\boldsymbol {\theta }},\mathbf {y} )$ and samples from marginal distributions. The estimated ratio satisfies $p({\boldsymbol {\theta }}\mid \mathbf {y} )\propto q_{\phi }({\boldsymbol {\theta }},\mathbf {y} )$, enabling posterior sampling via MCMC or rejection sampling. NRE avoids the need to normalize density estimators and can be more robust to high-dimensional observations, but requires sampling algorithms at inference time.

First neural density estimators used in SBI included autoregressive models and normalizing flow-based architectures. Recent advances use the flexibility of diffusion models which enables post-hoc adaptation of the approximator to a specific inference scenario such as changing priors.

Amortized Bayesian inference

A common feature of neural SBI is amortized Bayesian inference (ABI), in which an inference model is trained once on a set of simulations and then reused to perform inference for many future observations from the same simulator. ABI shifts computation from repeated per-dataset inference to an up-front training cost. In contrast, sequential inference methods adapt the simulation strategy iteratively based on intermediate posterior estimates. Sequential approaches generate simulations from progressively refined proposal distributions centered on regions of high posterior probability, which can improve sample efficiency when inference is needed for a single observation. Sequential methods trade the reusability of amortized methods for reduced simulation budgets and tighter posterior approximations in focused inference tasks.

Diagnostics

Common diagnostics for simulation-based inference include posterior predictive checks and simulation-based calibration (SBC), which assess whether inferred posteriors are consistent with the assumed data-generating process. The core idea of SBC is to repeatedly simulate parameters and data from the generative model, run the inference procedure, and compute the rank of the ground-truth parameter within samples from the inferred posterior. If the inference procedure is well calibrated, these ranks follow a uniform distribution. SBC is therefore a necessary condition for calibration, but it does not by itself guarantee that posterior approximations are accurate for all observations or all aspects of the distribution. Posterior predictive checks instead focus on whether data generated from the fitted model resemble the observed data under relevant test statistics or summary features, and can reveal model misspecification even when posterior calibration appears satisfactory.

Classifier two-sample tests (C2ST) are discriminative goodness-of-fit procedures which compare two distributions by training a classifier to distinguish samples drawn from each of them. In simulation-based inference, classifier-based diagnostics can be used to assess the quality of posterior approximations, by testing whether samples from the inferred posterior are statistically indistinguishable from samples generated by the reference procedure under the assumed model. Local classifier two-sample tests (L-C2ST) specialize this idea to provide observation-specific diagnostics for posterior approximations for a fixed observation. Unlike global calibration tests such as SBC, L-C2ST can reveal localized posterior failures, for example overly narrow uncertainty, shifted modes, or systematic mismatches concentrated in particular regions of the observation space.

Applications

Simulation-based inference is used in scientific domains where mechanistic simulators exist but analytic likelihoods are difficult to obtain, including:

• computational biology and population genetics
• neuroscience and cognitive science
• astronomy and cosmology
• particle physics
• epidemiology

The computational cost of SBI methods is dominated by simulator evaluations rather than inference algorithm overhead. Different approaches require vastly different simulation budgets depending on their strategy.

Advantages

Common advantages of SBI include:

• Bayesian uncertainty quantification via posterior distributions
• applicability to complex mechanistic simulators without tractable likelihoods
• flexibility for high-dimensional and structured observations
• fast inference after training in amortized settings

Limitations

Limitations of SBI methods include:

• potentially high computational cost due to simulation requirements
• sensitivity to simulator mismatch (model misspecification)
• extrapolation risks when simulations do not cover the regimes relevant to real observations

Software

Several software packages implement simulation-based inference (SBI), particularly neural and amortized methods. General-purpose toolkits typically provide implementations of neural posterior estimation, neural likelihood estimation, and neural ratio estimation, and are designed to integrate with simulators to generate training data for inference networks.

• BayesFlow is a Python library, supporting PyTorch, TensorFlow and Jax, for state-of-the-art amortized Bayesian inference with neural networks, focusing on simulation-based workflows in which an inference network is trained on simulated parameter–observation pairs and then reused for fast posterior inference.

• sbi is a Python (PyTorch) toolbox for simulation-based Bayesian inference, providing state-of-the-art neural SBI algorithms and workflows for posterior inference using simulation-trained estimators.

• sbijax implements simulation-based inference methods in the JAX ecosystem.

For classical approximate Bayesian computation tools, see Approximate Bayesian computation#Software.

See also

• Approximate Bayesian computation
• Bayesian inference
• Inverse problem
• Uncertainty quantification
• Markov chain Monte Carlo