on 18-Jun-2020 (Thu)

An excellent way to make code more maintainable is through loose coupling. As far back as 1994, when the Gang of Four wrote Design Patterns, this was already common knowledge: 2 Program to an interface, not an implementation.

for MCMC, we allow this candidate generating density (or proposal density) to depend upon the current state of the chain, and we denote it by q(x^t , y)

The optimal form for the acceptance function, in the sense that suitable candidates are rejected least often and computational efficiency is maximised, is given by .

The Metropolis Hastings algorithm has the advantage over the Gibbs Sampler, in that it is not necessary to know all of the conditional distributions, we need only simulate from q, which we can choose arbitrarily. Moreover, and this can be of crucial importance , we only need to know π up to proportionality, since any constants of proportionality cancel in the numerator and denominator of the calculation of α. The price for this simplicity is that if q is poorly chosen, then the numbe r of rejections can be high, so that the efficiency of the procedure can be low.

The single-update Metropolis Hastings algorithm is the Metropolis Hastings algorithm, updating variables one-at-a-time.

the Gibbs Sampler is a special case of the single-update Metropolis Hastings algorithm

Given a set of Markov kernels, each with invariant distribution π , it can be shown that, under weak conditions on the kernels, a hybrid algorithm consisting of mixtures or cycles of these kernels will produce a chain with the desired stationary distribution.

In a mixture of Markov kernels, probabilities are specified and, at each step, one of the kernels is selected according to these probabilities

In a cycle of Markov kernels, each of the kernels are used in a pre-determined sequence which is repeated until the sampler is stopped