Bayesian Statistics II

Overview

You will learn the basics of Bayesian Statistics

Goals

• Learn about Baysian Model Selection

• Learn about Variational Inference

• Learn about Optimization and Stochastic Gradient Descent

• Learn about the Cross Validation method of model training, validation and testing

Lecture Materials

• Slides

• Bayesian Model Selection

• Variational Inference

• Optimization

• Cross Validation

Homework Assignment

• Homework 04: Metropolis-Hastings and Cross Validation

Supplemental Readings

• C. Maes, An introduction to the theory of Markov processes mostly for physics students

• Example of dependence without correlation

• Conditional Independence

• Inverse Problem

• Brownian Motion

• Hamiltonian Mechanics

• Cannonical Distribution

• Hamiltonian MC

• Autocorrelation

• Convex Functions

• Jensen’s Inequality

• Finite Difference Equations

• Automatic Differentiation

• Rosenbrock function

• Nelder-Mead method

• Conjugate Gradient Method

• Newton’s CG method

• Powell’s method

• BFGS method

• Stochastic Gradient Descent

• Adam Optimizer

• Softmax Function