ST3456

ST3456: Modern Statistical Methods II

Module Code: ST3456

Module Title: Modern Statistical Methods II

Cohorts: Maths

Week range: 21-26, 28-32

Semester: S2 (Hilary)

Total hours: 33

Timetable:

- Monday, h. 11-13, room 1.07

- Wednesday, h. 11, room LB01

Textbook:

  • Rubinstein & Kroese, "Simulation and Monte Carlo Method"

Other references:

  • Ross, "Simulation"

  • Robert & Casella, "Monte Carlo statistical methods"

  • Robert & Casella, "Introducing Monte Carlo methods with R"

  • Devroye, "Non-uniform random variate generation".

Lecture notes used during the academic year 2016/17 are available upon request.

Syllabus (lecture by lecture)

16/01/17 (2h): Motivating problems: numerical integration and Bayesian inference. Example on Weibull model for leukemia data.

18/01/17 (1h): Inverse transform method with proof. Examples: Exponential, Gamma with integer shape parameter, order statistics.

23/01/17 (2h): Inverse transform method for discrete random variables. Box-Muller algorithm. Acceptance-Rejection (AR) method with proof.

25/01/17 (1h): Properties of AR method. Example: AR method for gamma distribution with exponential envelope (first part on admissible parameters).

01/02/17 (1h): Example: AR method for gamma distribution with exponential envelope (second part on optimal choice of the parameter of the exponential). Measures of the quality of an estimator: mean squared error (MSE) and relative error (RE).

06/02/17 (2h): Example: Monte Carlo estimation of the probability of rare events. The idea behind variance reduction techniques. Use of common and antithetic variables. Example: estimation of e by using antithetic variables.

13/02/17 (2h): Using control variables. Review of conditional expectation. Conditional Monte Carlo.

15/02/17 (1h): Example "estimating pi" to compare different methods for variance reduction. (Matlab code - the code is not part of the course program, i.e. not examinable)

20/02/17 (2h): Stratified sampling. Variance reduction in case of stratum sample sizes proportional to weights (with proof). Neyman allocation for optimal sample sizes (with proof).

22/02/17 (1h): Importance sampling. Observations on how to choose the candidate density. Example: estimating P(Z>4.5) with Z Standard Normal.

06/03/17 (2h): Importance sampling with exponentially tilted instrumental densities. Bounds on the estimator for rare probabilities and optimal choice of the tilting parameter. Example.

08/03/17 (1h): Review of Markov chains (first part).

13/03/17 (2h): Review of Markov chains (second part).

15/03/17 (1h): Example: Markov Chain converging to Poisson distribution (videos 1 and 2). Metropolis-Hastings algorithm (discrete state space).

20/03/17 (2h): Metropolis-Hastings algorithm (continuous state space). Independent Metropolis-Hastings.

22/03/17 (1h): Example: comparison Independent Metropolis-Hastings vs Acceptance-Rejection. Random walk Metropolis-Hastings. Example: simulating bimodal bivariate density with random walk Metropolis-Hastings.

27/03/17 (2h): Acceptance rate in Metropolis-Hastings. Two-stage Gibbs sampling algorithm. Detailed balanced equation for component-wise Gibbs sampling. Two-stage Gibbs sampling with random scan. Example: Gibbs sampling for bivariate density.

29/03/17 (1h): Multi-stage Gibbs sampling. Connection between Gibbs-sampling and Metropolis-Hastings. Example (continued): comparison of Metropolis-Hastings and Gibbs sampling for bivariate density.

03/04/17 (2h): Monitoring the convergence of MCMC methods. Example: Bayesian analysis of coal mine data.

05/04/17 (1h): Solution of the mock exam paper.

Videos

Video 1:

Video 2:

Video 3:

Video 4:

Video 5:

Video 6: