Bayesian approaches to discovery and inference with non-linear causal models

Causal discovery and inference from observational data is an essential problem in statistics posing both modeling and computational challenges. Bayesian network models provide a rigorous approach to the problem by compactly describing the joint distribution and the causal relationships through a graph. When the underlying graph is unknown, structure learning methods are used to estimate it from the available data. Such a task poses considerable challenges, but is key to describing data-generating mechanisms in many complex applications. This thesis presents new developments in structure learning and causal inference, with a focus on computational efficiency and non-parametric, Bayesian approaches for observational data.
Firstly, we introduce a new constraint-based structure learning algorithm based on the popular PC algorithm. The new method, called the dual PC algorithm, leverages the inverse relationship between covariance and precision matrices. By exploiting block matrix inversions it can also perform tests on partial correlations of complementary (or dual) conditioning sets. The dual PC algorithm proceeds by first considering marginal and full-order conditional independence relationships and progressively moving to central-order ones. Simulation studies show that the dual PC algorithm outperforms the classic PC algorithm both in terms of run time and in recovering the underlying network structure. We also study the effects of deviations from Gaussianity on the performance of the classical and dual PC algorithms, and propose alternative approaches when consistency no longer holds.
Next, we address the problem of learning the structure of Gaussian Process Networks (GPNs), a class of Bayesian networks which employ Gaussian processes as priors for the conditional expectation of each variable given its parents. We adopt a Bayesian approach, accounting for uncertainty in the graphical estimate via a posterior distribution over structures. We show how Monte Carlo and Markov chain Monte Carlo methods can be used to sample from the graph posterior distribution, allowing accurate posterior inference in high-dimensional cases. Our method outperforms state-of-the-art algorithms in recovering the graphical structure of the network in various non-linear simulation settings. The proposed method is also shown to provide an accurate approximation of the network's posterior distribution.
Finally, we consider the problem of the Bayesian estimation of the effects of hypothetical interventions in the GPN model. We detail how to perform causal inference on GPNs by simulating the effect of an intervention across the whole network and propagating the effect of the intervention on downstream variables. A simpler computational approximation can be derived by estimating the intervention distribution as a function of local variables only, modeling the conditional distributions via additive Gaussian processes. We extend both frameworks beyond the case of a known causal graph, incorporating uncertainty about the causal structure using the previously developed Bayesian structure learning methods. Simulation studies show that our approach is able to identify the effects of hypothetical interventions with non-Gaussian, non-linear observational data and accurately reflect the posterior uncertainty of the causal estimates. Finally we compare the results of our GPN-based causal inference approach to existing methods on a dataset of A. thaliana gene expressions.