-
Ensemble samplers can sometimes work in high dimensions
A few years ago Bob Carpenter wrote a fantastic post on how why ensemble methods cannot work in high dimensions. He explains how high dimensions proposals that interpolate and extrapolate among samples are unlikely to fall in the typical set, causing the sampler to fail. During my PhD I worked on sampling problems with infinite-dimension parameters (ie: functions) and intractible gradients. After reading Bob’s post I always wondered if there was a way around this problem.
-
How to add a progress bar to JAX scans and loops
JAX allows you to write optimisers and samplers which are really fast if you use the
scan
orfori_loop
functions. However if you write them in this way it’s not obvious how to add progress bar for your algorithm. This post explains how to make a progress bar using Python’sprint
function as well as using tqdm. After briefly setting up the sampler, we first go over how to create a basic version using Python’sprint
function, and then show how to create a nicer version using tqdm. You can find the code for the basic version here and the code for the tqdm version here. -
MCMC in JAX with benchmarks: 3 ways to write a sampler
This post goes over 3 ways to write a sampler using JAX. I found that although there are a bunch of tutorials about learning the basics of JAX, it was not clear to me what was the best way to write a sampler in JAX. In particular, how much of the sampler should you write in JAX? Just the log-posterior (or the loss in the case of optimisation), or the entire loop? This blog post tries to answer this by going over 3 ways to write a sampler while focusing on the speed of each sampler.
-
Implementing natural numbers in OCaml
In this post we’re going to implement natural numbers (positive integers) in OCaml to see how we can define numbers from first principle, namely without using OCaml’s built in
Integer
type. We’ll then write a simple UI so that we have a basic (but inefficient) calculator. You can find all the code for this post on Github. -
Testing MCMC code: the prior reproduction test
Markov Chain Monte Carlo (MCMC) is a class of algorithms for sampling from probability distributions. These are very useful algorithms, but it’s easy to go wrong and obtain samples from the wrong probability distribution. What’s more, it won’t be obvious if the sampler fails, so we need ways to check whether it’s working correctly.