In this blog post I will share with you the materials that I found most useful when I started doing some Bayesian Optimisation in my research. Bear in mind, I am a Chemist by training, so I approached this topic from a non-mathematical background (my eyes have to be persuaded to look at mathematical equations). Out of all the materials I have come across, I found these to be the most accessible.
(1) Nando de Freitas’ lecture on Bayesian optimization and multi-armed bandits
This is a YouTube video of a University of British Columbia computer science lecture by Nando de Freitas on Bayesian Optimisation and multi-armed bandits. It manages to introduce the fundamentals of Bayesian Optimisation at a very gentle pace and explains each equation in detail (I like his annotations especially). It is very easy to follow and well worth the 1 h 20 mins – he is an excellent lecturer.
(2) GPyOpt IPython notebook examples
Anything interactive is a thumbs up from me. GPyOpt has a great manual, including several example notebooks, all well commented and easy to follow. I like their simple weather map example, which is a very accessible problem and not field specific. Admittedly, I have referenced this analogy in several of my presentations which introduces Bayesian Optimisation.
(3) These two papers:
Frazier, P. I. (2018). A Tutorial on Bayesian Optimization, (Section 5), 1–22.
Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., & de Freitas, N. (2016). Taking the Human Out of the Loop: A Review of Bayesian Optimization. Proceedings of the IEEE, 104(1), 148–175.
If you want to read some literature, try these two; the first is a tutorial and the second is a review. Both go from the basics to the more advance.
Finally, thanks to these materials, I am surely a bit wiser in Bayesian Optimisation but I definitely have still a lot to learn (if you have any more nice materials please point them my way)! Also a big thanks to fellow OPIG member, Lucian, for introducing many (most) of these materials to me (it also helps enormously if you have someone who does know about Bayesian Optimisation sitting in the same office as you)!
