Eindhoven University of Technology
Scientific modeling concerns a continual search for better models for given data sets. This process can be elegantly captured in a Bayesian (probabilistic) inference framework. ForneyLab is an open source probabilistic programming toolbox that enables largely automated design loops by deriving fast, analytic algorithms for message passing-based approximate Bayesian inference. Probabilistic programming extends conventional programming languages with the facility to compute rationally with random variables through probability calculus. ForneyLab is a Julia package that allows the user to specify a probabilistic model and pose inference problems on those models. In return, FL automatically constructs a Julia program that executes a message passing-based approximate inference procedure. In conjunction with a specification of (possibly streaming) data sources, the inference program can be executed to fit the model to the data. Additionally, ForneyLab provides a measure of the performance of the model fit, thus facilitating comparison to alternative models. Typical applications include the design of dynamic models for (Bayesian) machine learning systems, statistical signal processing, computational neuroscience, etc. More specifically, a design cycle in ForneyLab consists of three phases. First, in the build phase, the end user specifies a (probabilistic) model. Through the use of macros, the model is specified in a domain-specific syntax that strongly resembles notational conventions in alternative probabilistic programming languages. Usually, even complex model specifications fit on less than one page of code. Under the hood, ForneyLab builds a Forney-style factor graph (FFG), which is a computational network representation of the model. A strong feature of the FFG formalism includes its extremely modular make-up, which allows re-use of computational inference primitives. The choice for FFG-based model specifications (and consequently, message passing-based inference procedures) is where ForneyLab differs from competing probabilistic programming languages (aside from our choice for a native Julia realization). Next, in the schedule phase, the user specifies the inference problem. This is usually encoded by a few lines of code. ForneyLab then automatically derives a message passing algorithm that, when executed, computes the posterior marginal probability over the desired variables. The generated message passing code may contain thousands of code lines, depending on the size of the model. A clear asset of message passing-based inference algorithms is that they are comprised of many cheap and analytical updates that can be re-used across models, and furthermore can potentially be implemented on dedicated (parallel) hardware configurations. This contrasts to modern sampling-based approaches to inference, such as Markov chain Monte Carlo and Automatic Differentiation Variational Inference, which usually require massive computational resources. In the final infer phase, ForneyLab parses and executes the automatically generated inference program. For the user, this action is initiated by one statement. Additionally, a separate function can be generated to evaluate the model fit, which provides insights on model quality and algorithm convergence during inference. ForneyLab relies heavily on Julia’s meta-programming functionality. Not only does it use macros for the model specification, but the main output is also a Julia program by itself. This flexibility, together with benefits of the modular FFG approach, makes it a powerful tool for a scientist or engineer who wants to develop models for a given data set.
Thijs van de Laar received his MSc degree in natural sciences from the Radboud University Nijmegen in 2010. Currently he works as a PhD student in the BIASlab group (http://biaslab.org) at the Electrical Engineering department at Eindhoven University of Technology (TU/e). He is interested in Bayesian machine learning for personalized audio processing.
