Joe KroeseNavigate back to the homepage

A Guided Route through Learning Bayesian Statistics

Joe Kroese
April 27th, 2020 · 3 min read

If I catch myself sharing the same advice twice, I write it down somewhere.

Whilst my frequentist mind would have felt that Bayesian statistics, with its priors and likelihoods, was too conceptual to use in practice, it has proven the opposite. I’ve been blown away by the flexibility of modern Bayesian methods to deal with missing data, combine multiple data source without even mentioning how uncertainty is quantified.

I’m keen to share how to build up the skills necessary to start using it in analysis. Here is my suggestion.

Step 0: Meet the Family

Meet the Family

For me, while frequentist statistics relies heavily on heuristic rules of when to use certain statistical tests and methods, Bayesian analysis involves stating everything you know and then letting the model work itself out. As such, a lot of it is about a different mindset. To help get into that, it helps to surround yourself with voices with experience of these alternative ways of seeing.

Separately, reading how others are using Bayesian statistics will serve as inspiration when you get lost in the murk of exploring Bayesian statistics.

Step 0.0 Follow some Bayesian blogs.

Follow statmodeling. It is predominantly written by Andrew Gelman but also features other nice people associated with Columbia University and Stan (a Bayesian programming language). The posts here are great and the comments are even better.

I would love to hear of more blogs worth following, so please contact me if you have suggestions.

I find that statisticians don’t put design at top of their list so I would recommend using an RSS feed gatherer, like Feedly to view the different blogs.

Step 0.1 Academic Twitter

Join academic Twitter. You can say what you want about Twitter – and I do – but unfortunately, a lot of the best conversations are there. A good starting place is following rlmcelreath and betanalpha. From there, you’ll be able to find the Bayesians in your field or demographic. For example, queer Bayesian ecology Twitter is surprisingly busy. If you’re interested, my username is joekroese.

Step 1: Building a Strong Foundation

Strong Foundations

Now you’ve done the easy part, it’s time to knuckle down: you have to learn the fundamental concepts of Bayesian statistics.

Step 1.0 Statistical Rethinking

For most people, the best way to do this will be reading and working through McElreath’s Statistical Rethinking. This is a beautiful book. It teaches the basics of Bayesian statistics up to and a bit past using multilevel models. As a bonus it comes with a refreshing philosophical view from McElreath’s background as an anthropologist.

The book uses an R package made for the teaching of the book called rethinking. It is surprisingly powerful. It balances the dual needs of ensuring you understand the content without getting lost in technical details. Neither brms nor Stan can do this very well. (brms obscures the content and Stan requires a lot of technical skills to get started with the concepts.)

Optional Step 1.1: BDA as an Accompanying Text

If you come from an academic maths background, I would recommend having Bayesian Data Analysis on hand. I found some of the key definitions a little lacking in Statistical Rethinking, at least from the point of view of those coming from a certain background. BDA, as it is warmly known in the community, is perfect for filling in these gaps with precise mathematical definitions.

Step 2: Learn the Tools


Step 0 and 1 introduced you to the community and got you started with the concepts. Now it’s time to learn the software that most people are using for Bayesian statistics.

Step 2.0 Statistical Rethinking Recoded

This is the sequel we needed.

Matrix Reloaded

Statistical Rethinking Recoded rewrites Statistical Rethinking in the framework for modern data science in R, Tidyverse, and a very powerful package for communicating between Stan and R called brms. Working through this book will show you how to use all the concepts you learnt in Statistical Rethinking but with the tools that applied statisticians use.

Step 3: Onwards and Upwards

From here, you’ll be in a strong place to pursue the work that interests you. Whether that’s in ecology or political science, there are active communities using Bayesian statistics.

To support you in your next steps, the Stan community are welcoming for all stages of people using Stan. A list of other useful resources is here.

Enjoy the journey!

More articles from Joe Kroese

Gratitude Machine

An AI learning how to be thankful

November 1st, 2019 · 2 min read

Dance Montage

Using AI to create choreographies across the history of film and YouTube

April 3rd, 2019 · 1 min read
© 2017–2020 Joe Kroese
Link to $ to $ to $ to $