By: Nathan Bishop

Nathan Bishop — Tue, 18 Sep 2012 00:17:34 +0000

Thank you for your answer. My background is physics so my questions might sound trivial, but what is the insight in putting u equal to F^{-1}(x_j^{old})? Can I use a linear combination of F^{-1}(x_j^{old}) and F^{-1}(x_j^{older})?

By: Radford Neal

Radford Neal — Mon, 17 Sep 2012 13:40:45 +0000

No. That wouldn’t make sense, since x needn’t be bounded by 0 and 1, and so may not be a valid argument of F inverse. Plus u is supposed to be in [0,1).

By: Nathan Bishop

Nathan Bishop — Mon, 17 Sep 2012 02:47:12 +0000

Do you mean setting u to F^{-1}(x_j^{old}), where x is the previous value?

By: Radford Neal

Radford Neal — Thu, 17 May 2012 16:48:14 +0000

That's an interesting question. I've just tried a 10-dimensional truncated multivariate normal, with mean vector zero and a covariance matrix built as follows (in R):


X =  rbind( c(1, 3, 3, 2, 1, 0, 1),
            c(2, 1, 3, 2, 1, 1, 1),
            c(2, 1, 2, 2, 1, 3, 1),
            c(1,-1,-2,-3,-1,-2,-1),
            c(0, 3, 0, 3, 1, 1, 0),
            c(2, 0,-2,-3, 0,-2,-1),
            c(2, 0,-2,-3, 2, 2,-1),
            c(1, 3, 2,-3, 2, 2,-1),
            c(0, 2, 0, 0, 5, 0, 0),
            c(0, 0, 3, 0, 0,-4, 0))
cov = X %*% t(X) + diag(10)

I truncated this distribution to minimum values of -10 for all coordinates and maximum values of 5, 6, ..., 14. I simulated 250 parallel Gibbs sampling chains, started from points drawn uniformly from [-1,1]¹⁰, in several ways. First, using standard Gibbs sampling, with the 250 chains being independent. Second using permutation Gibbs sampling, with the same random values for s₀, s₁, ... for all chains (but different initial states). Finally, I tried fixing all the s_i to a single value, either 0.0073, 0.0187, 0.0518, or 0.1377. I simulated 400 burnin iterations for each chain, followed by 10000 iterations taken to be from equilibrium. Using these 250x10000 states, I computed estimates of the expected values of each coordinate (which aren't zero, due to truncation) and of the expected values of the squares of the coordinates, along with estimated standard errors for these estimates, based on the variation over the 250 chains. The results are plotted here. As you can see from the first and third plots, all the methods produce very consistent results. The standard errors of the methods differ, however. Standard Gibbs sampling and permuation Gibbs sampling with random s_i are very similar, but the standard errors for the coordinates are smaller by roughly a factor of two when s_i is fixed at 0.0073, corresponding to about a four times efficiency advantage over standard Gibbs sampling. Fixing the s_i at 0.0187 gives a smaller advantage, and the advantage is smaller still when the s_i are fixed at 0.0518. When the s_i are fixed at 0.1377, the results are very close to standard Gibbs sampling and permutation Gibbs sampling with random s_i. The standard errors for the estimates of the expected values of the squares of the coordinates are all about the same, except that they are sometimes bigger when the s_i are fixed at 0.0073. So it seems that fixing all the s_i to 0.0187 gives clearly better results than standard Gibbs sampling.

By: Mark Johnson

Mark Johnson — Wed, 16 May 2012 12:02:42 +0000

It’s very interesting that this deterministic modification of Gibbs sampling works at all, let alone so well, on this small example. But deterministic quadrature rules can be efficient in low dimensions. How well do these methods work in higher dimensions?

Comments on: Non-random MCMC

By: Nathan Bishop

By: Radford Neal

By: Nathan Bishop

By: Radford Neal

By: Mark Johnson