ST3456: Modern Statistical Methods II
Module Code: ST3456
Module Title: Modern Statistical Methods II
Week range: 21-26, 28-32
Semester: S2 (Hilary)
Total hours: 33
- Monday, h. 11-13, room 1.07
- Wednesday, h. 11, room LB01
Rubinstein & Kroese, "Simulation and Monte Carlo Method"
Robert & Casella, "Monte Carlo statistical methods"
Robert & Casella, "Introducing Monte Carlo methods with R"
Devroye, "Non-uniform random variate generation".
Lecture notes used during the academic year 2016/17 are available upon request.
Syllabus (lecture by lecture)
16/01/17 (2h): Motivating problems: numerical integration and Bayesian inference. Example on Weibull model for leukemia data.
18/01/17 (1h): Inverse transform method with proof. Examples: Exponential, Gamma with integer shape parameter, order statistics.
23/01/17 (2h): Inverse transform method for discrete random variables. Box-Muller algorithm. Acceptance-Rejection (AR) method with proof.
25/01/17 (1h): Properties of AR method. Example: AR method for gamma distribution with exponential envelope (first part on admissible parameters).
01/02/17 (1h): Example: AR method for gamma distribution with exponential envelope (second part on optimal choice of the parameter of the exponential). Measures of the quality of an estimator: mean squared error (MSE) and relative error (RE).
06/02/17 (2h): Example: Monte Carlo estimation of the probability of rare events. The idea behind variance reduction techniques. Use of common and antithetic variables. Example: estimation of e by using antithetic variables.
13/02/17 (2h): Using control variables. Review of conditional expectation. Conditional Monte Carlo.
15/02/17 (1h): Example "estimating pi" to compare different methods for variance reduction. (Matlab code - the code is not part of the course program, i.e. not examinable)
20/02/17 (2h): Stratified sampling. Variance reduction in case of stratum sample sizes proportional to weights (with proof). Neyman allocation for optimal sample sizes (with proof).
22/02/17 (1h): Importance sampling. Observations on how to choose the candidate density. Example: estimating P(Z>4.5) with Z Standard Normal.
06/03/17 (2h): Importance sampling with exponentially tilted instrumental densities. Bounds on the estimator for rare probabilities and optimal choice of the tilting parameter. Example.
08/03/17 (1h): Review of Markov chains (first part).
13/03/17 (2h): Review of Markov chains (second part).
15/03/17 (1h): Example: Markov Chain converging to Poisson distribution (videos 1 and 2). Metropolis-Hastings algorithm (discrete state space).
20/03/17 (2h): Metropolis-Hastings algorithm (continuous state space). Independent Metropolis-Hastings.
22/03/17 (1h): Example: comparison Independent Metropolis-Hastings vs Acceptance-Rejection. Random walk Metropolis-Hastings. Example: simulating bimodal bivariate density with random walk Metropolis-Hastings.
27/03/17 (2h): Acceptance rate in Metropolis-Hastings. Two-stage Gibbs sampling algorithm. Detailed balanced equation for component-wise Gibbs sampling. Two-stage Gibbs sampling with random scan. Example: Gibbs sampling for bivariate density.
29/03/17 (1h): Multi-stage Gibbs sampling. Connection between Gibbs-sampling and Metropolis-Hastings. Example (continued): comparison of Metropolis-Hastings and Gibbs sampling for bivariate density.
03/04/17 (2h): Monitoring the convergence of MCMC methods. Example: Bayesian analysis of coal mine data.
05/04/17 (1h): Solution of the mock exam paper.