Research Interests
I am internationally recognized as one of the experts on computational complexity aspects of Bayesian networks, which was also the topic of my PhD research. After my PhD period I specialized in the complexity of approximating the most probable explanation for observed variables in such networks. I proposed a new heuristic, so-called Most Frugal Explanation, for this computationally intractable problem. In addition, I proposed new parameterized complexity classes that capture the notion of efficient randomized computations. In contrast to the traditional parameterized complexity class FPT, these new classes parameterize the probability of answering correctly, rather than computation time.

In recent years I became interested in (resource-bounded) probabilistic inferences in the brain, in particular in the context of the "Predictive Processing" account in neuroscience. Together with researchers at the Donders Institute for Brain, Cognition, and Behaviour, I contributed computational models and mathematical formalizations to this account. In particular, I identified the level of detail of predictions and the trade-off between making precise predictions and predictions that are likely to be correct as crucial aspects of the theory. I also contributed computational models of various mechanisms of prediction error minimization and studied the computational complexity of sub-processes in the Predictive Processing account.

In addition to our more 'traditional' work in Probabilistic Graphical Models (such as the parameterized tractability of approximate Bayesian inferences and the search for novel approximation techniques) our current research focuses on the nature of stochastic computations in the brain and in brain-inspired hardware. For example, we are interested in the question how to formally model extension of PGMs with new variables or values of variables and how how the generative models are developed in infancy. A novel interest is in resource-bounded approximate computations in the brain, in particular in computational (in-)tractability relative to the brain's resources, such as in neuromorphic hardware. The fundamental work above is complemented with more applied work focusing on explainability in clinical decision support systems and on energy-lean neuromorphic computing in physics and the semiconductor industry.

Sat2Inf project (2022-2026; PhD candidate: vacancy)
Bayesian inference is an intractable (NP-hard) problem even when only an approximate solution is sought, implying that well-known approximation techniques for Bayesian inference include variational Bayes, Metropolis-Hasting sampling, and likelihood weighting only work well on a subset of problem instances, and cannot give a guaranteed quality of approximation in general. This limits the applicability of Bayesian networks for real world applications.

In this PhD project we investigate a new approach towards approximate Bayesian inference, i.e., we translate an inference problem to a weighted satisability instance, apply approximation strategies to give an approximate count of the (weighted) number of models of the instance, and then translate the solution back to the inference problem. This approach may open up new avenues as it allows for a new class of approximation strategies based on hashing rather than sampling or model simplification to approximately count models. Currently, however, the state-of-the-art techniques are not yet well suited for the instances that arise from the translation from a Bayesian network to a weighted satisability problem. In this project we study how this translation can adjusted such that hashing approaches work, and study both experimentally and by formal parameterized complexity analysis whether this allows for a novel sub-set of Bayesian inference problems that can be tractably approximated.

This project is funded by an NWO exact sciences M1 open competition grant awarded to Johan Kwisthout (M21.136).

M++ project (2020-2024; PhD candidate: Erwin de Wolff; co-supervision with Iris van Rooij)
An influential theoretical position in cognitive neuroscience is that the brain is a 'prediction machine' that uses internal generative models to guide perception and action. Integrating new information with existing internal models is a crucial 'operation' that underlies learning and education, but also development, behaviour change, creativity, incident recovery, and drug rehabilitation. When cast into Bayesian terms (e.g., predictive coding) this integration translates into (non-parametric) changes to the structure of existing models, such as the introduction of new variables, the conditioning of an existing variable on a new contextual dependence, or the re-allocation of probability mass over variables. There is currently no formal framework that systematically addresses these operations, which seriously constrains scientists that aim to use Bayesian computational models to describe (neuro-)cognitive phenomena (and interpret data) that require structure changes in internal models. In this project we will develop such a formal framework to remedy this omission.

This project is funded by an internal DCC PhD-grant awarded to Johan Kwisthout (main applicant) and Iris van Rooij (co-applicant).

NALCA project (2018-2028; Postdoc: Leila Bagheriye, PhD candidate: Arne Diehl)
In traditional (Von Neumann) computer architectures computation and memory are physically separated by a (relatively slow) data bus, requiring excessive data transfer with every instruction cycle and high energy consumption as a consequence. This has accelerated the development of so-called neuromorphic architectures that, inspired by the structure of the brain, co-locate computation and memory in artificial (spiking) neural networks. This allows for a new way of thinking about algorithms and data and designing energy-efficient algorithms that encode information in time-dependent spiking behaviour. Theory, however, is now seriously lagging behind hardware innovations. We do not yet fully understand and utilize the potential and limitations of these new architectures. In this research project we lay the foundations for an urgently needed computational complexity framework for neuromorphic hardware focusing on energy as a vital resource. Foundational complexity-theoretical work is augmented by a parallel project, supported by Intel, where we translate abstract algorithms on generic computing devices to actual implementations on Intel’s new Loihi architecture, allowing us to validate and refine the theoretical models. Theoretical and practical research together will lay a firm foundation for the further development of energy-efficient computing to help reducing carbon footprints.

This project is partially funded by an Intel Neuromorphic Research Community grant awarded to Johan Kwisthout (applicant).

p-CABI project (2017-2021; PhD candidate: Nils Donselaar)
Computing posterior and marginal probabilities constitutes the backbone of almost all inferences in Bayesian networks. These computations are known to be intractable in general; moreover, it is known that approximating these computations is also NP-hard. These theoretical results are in stark contrast with common experience in practical randomized approximation algorithms that often -but not always!- are able to approximate large-scale inferences. Our understanding of when, why, and how randomized approximate inference can be tractable in Bayesian networks is currently still limited.

This is even more problematic as this hinders progress in other scientific disciplines, in particular cognitive science and neuroscience. The influential `Predictive Processing' account of cortical processes in neuroscience is explicitly built on the formal machinery of approximate Bayesian computations. There is an urgent need for a deeper theoretical understanding of these computations and the constraints that need to be imposed on their inputs such as to ensure tractability - a crucial prerequisite for neuronal implementation. However, recent advances in the area of parameterized complexity theory, where recently a randomized analogue of fixed-parameter tractability has been proposed, allow us to now tackle these questions.

In this project our goal is to thoroughly study these computations from different perspectives: studying structural properties of fixed-error randomized tractability ("Fundamentals"); designing and analysing efficient and predictable approximation algorithms that allow to trade off sampling and marginalizing in inference ("Engineering"); and studying the neuroscientific claim that predicting inputs based on generative models allows for efficient approximate computations by the brain ("Reverse Engineer").

This project is funded by an NWO exact sciences TOP grant awarded to Johan Kwisthout (612.001.601).