ST3456: Modern Statistical Methods II
Module Code: ST3456 (descriptor) Module Title: Modern Statistical Methods II (Simulation methods) Cohorts: Maths Week range: 0308, 1014 Semester: S1 (Michaelmas) Total hours: 33 (28 classes + 5 labs)
Timetable:  Monday, h. 1214, room LB01  Tuesday, h. 1011, room LB 1.07 (or ICT1 lab when labs are scheduled)
Textbook:  Lecture notes will be uploaded to this webpage
Other references:  Rubinstein & Kroese, "Simulation and Monte Carlo Method"
 Robert & Casella, "Monte Carlo statistical methods"
 Robert & Casella, "Introducing Monte Carlo methods with R"
 Devroye, "Nonuniform random variate generation"
Links:
Group project (password protected)  instructions (pdf), deadline: 07/01/2019
Lecture notes (password protected):  10/09/18: Section 1: motivating examples (pdf)
 11/09/18: Section 2.1: inverse transform method (pdf)
 17/09/18: Section 2.2: transformation methods (pdf)
 18/09/18: Section 2.3: acceptancerejection method (pdf)
 01/10/18: Section 3, 3.1: controlling the variance; common and antithetic variables (pdf)
 08/10/18: Section 3.2, 3.3.1: control variables; conditional expectation (pdf)
 15/10/18: Section 3.3.2: variance reduction by conditioning (pdf)
 15/10/18: Lecture notes first 6 weeks: Sections 1, 2 and 3 (pdf)
 12/11/18: Sections 4.1 and 4.2: essential of Markov chains; MetropolisHastings for discrete statespace (pdf)
 19/11/18: Sections 4.3 and 4.4: MetropolisHastings for continuous statespace; independent MH; random walk MH (pdf)
 26/11/18: Lecture notes, final version (pdf) (last updated 29/11/18, corrections appear in red and are listed here)
Problem sets (password protected):  24/09/18: problem set 1 (pdf), solutions (pdf)
 12/11/18: problem set 2 (pdf), solutions (pdf)
Slides (password protected):  12/11/18: Example: random walk with limiting Poisson distribution (ppt)
 19/11/18: Example: independent MH vs AcceptanceRejection; Example: random walk MH and role of the parameter sigma (ppt)
 26/11/18: Example: Gibbs sampling vs MetropolisHastings; Example: diagnostic tools for MCMC (ppt)
Labs (password protected):  25/09/18: Lab 1  instructions (pdf), R script (R) and solutions (R)
 02/10/18: Lab 2  instructions (pdf), R script (R) and solutions (R)
 09/10/18: Lab 3  instructions (pdf), R script (R) and solutions (R)
 16/10/18: Lab 4  instructions (pdf), R script (R) and solutions (R)
 06/11/18: Lab 5  instructions (pdf), R script (R) and solutions (R)
 13/11/18: Lab 6  instructions (pdf), R script (R) and solutions (R)
 20/11/18: Lab 7  instructions (pdf), R script (R) and solutions (R)
Syllabus (lecture by lecture):  10/09/18 (2 h): motivating examples: Monte Carlo integration, Bayesian Inference, analysis of leukaemia data.
 11/09/18 (1 h): inverse transform method (with proof), ITM for exponential random variables (example 2.5), max and min of IID random variables (first method).
 17/09/18 (2 h): max and min of IID random variables (ITM method), ITM for discrete random variable, Transformation methods, BoxMuller method.
Assignment: see example 2.7 (ITM for Gamma(n, lambda)).
 18/09/18 (1 h): acceptancerejection (AR) method, with proof.
 24/09/18 (2 h): geometric distribution for the number of needed iterations in AR, AR for densities known up to a constant, example: optimal choice of exponential envelope for gamma target density.
 25/09/18 (1 h): lab 1  ITM for discrete random variables (Binomial and Poisson).
 01/10/18 (2 h): Monte Carlo estimation of expected value, mean squared error (MSE) and relative error, principle of variance reduction, estimating probability of rare event via crude Monte Carlo (example 3.1), antithetic variables (example 3.2), common variables (example 3.3).
 02/10/18 (1 h): lab 2  ITM (Exponential) and AR (Gamma).
 08/10/18 (2 h): control variables, conditional expectation, conditional variance formula.
 09/10/18 (1 h): lab 3  Simulating Normals via AR and BoxMuller.
 15/10/18 (2 h): variance reduction by conditioning, Example: comparing methods in estimating pi.
 16/10/18 (1 h): lab 4  Integral estimation via simulation
 30/10/18 (1 h): importance sampling method, example: estimating the probability of the rare event {Z>4.5}, with Z standard normal.
 05/11/18 (2 h): exponentially tilted distributions for importance sampling, exponentially tilted Bernoulli distribution, example: estimating the probability of a rare event with tilted proposal.
 06/11/18 (1 h): lab 5  Importance sampling with exponentially tilted proposal.
 12/11/18 (2 h): Essentials of Markov chains, MetropolisHastings algorithm for discrete statespace, Example: Poisson simulation via MH
 13/11/18 (1 h): lab 6  MetropolisHastings for discrete statespace
 19/11/18 (2 h): MetropolisHastings for continuous state space, independent MH, independent MH vs acceptance rejection, randomwalk MH, acceptance rate.
 13/11/18 (1 h): lab 7  MetropolisHastings for continuous statespace
 26/11/18 (2 h): Twostage Gibbs sampling with systematic scan, twostage Gibbs sampling with random scan, multistage Gibbs sampling, connection between Gibbs sampling and MetropolisHastings, Diagnostic tools for MCMC (very briefly).
 27/11/18 (1 h): lab  work on group project
 28/11/18 (1 h): tutorial  overview of exam of May 2017, Problem 4 in problem set 2 on Gibbs sampling.
Videos:  18/09/18: acceptancerejection for bounded density on compact support:
 18/09/18: acceptancerejection for truncated Normal(0,1) with Exponential(1) envelope:
 05/11/18: Markov chain with Poisson limiting distribution and initial state x0=1:
Poisson_x01_histogram_traceplot.mov
 05/11/18: Markov chain with Poisson limiting distribution and initial state x0=22:
Poisson_x22_histogram_traceplot.mov
 19/11/18: Random walk MetropolisHastings for a bivariate and bimodal distribution, with standard deviation for the marginal normal proposal sigma=0.1:
 19/11/18: Random walk MetropolisHastings for a bivariate and bimodal distribution, with standard deviation for the marginal normal proposal sigma=1:
 19/11/18: Random walk MetropolisHastings for a bivariate and bimodal distribution, with standard deviation for the marginal normal proposal sigma=10:
 26/11/18: Gibbs sampling for a bivariate and bimodal distribution:
