Thank you so much for this post, I’ve found it really useful. I’m about to implement some method to calculate the marginal likelihood of an econometric model and now I see that the harmonic mean estimator is not an attractive solution. What do you think about Geweke’s (1999) modified harmonic mean estimator (p.46 here: http://www.tandfonline.com/doi/pdf/10.1080/07474939908800428)?

Thank you for your help in advance!

Best,
Greg

#this is an example that kills me
a=matrix(1,4,4)
# transpose of the first row is a 4×1 COLUMN matrix
t(a[1,])

sadly, first R removes the dim attr, then does a transpose on the vector (which by some bizarre logic is possible with no dim attr) and gives you a 1×4 ROW matrix.

Horrible. However this is easily circumvented by employing drop=FALSE defensively throughout all your R code. Until… you need to work with arrays with more than 2 dimensions.

a=array(1,dim=c(1,2,10))
#once again R drops the dim attr and we get a row
t(a[,,1])
#that’s ok, let’s use drop=FALSE
t(a[,,1,drop=FALSE])
#oh no, a is not a matrix so we get an error
#–which actually makes sense unlike the behavior for t() on a vector with no dim attr

But what do we do to get a[,,1] as a matrix so we can take the transpose?

#this works but is computationally 3x as expensive as needed
b=a[,,1]
dim(b)=c(dim(a)[1],dim(a)[2])
t(b)

#this is less expensive but ugly
matrix(a[,,1],dim(a)[2],dim(a)[1],byrow=TRUE)

#by the way, this does not work for getting the correct transpose of a matrix
b=a[,,1]
dim(b)=c(dim(a)[2],dim(a)[1])

My solution was to write my own utility function for subsetting 3D matrices.

As I come from an optimisation background, BFGS is very appealing, but not so simple to see how it can be used in an MCMC scheme.

I do all this for optimisation of complex engineering designs.

The Burda and Maheu paper looks interesting, but I have been advised by others (who are of good repute) that it may not be reversible, so I would treat it with caution.

DREAM and its variants can be very useful. It is a nice idea.

Do you have any thoughts about “BAYESIAN ADAPTIVELY UPDATED HAMILTONIAN MONTE CARLO WITH AN APPLICATION TO HIGH-DIMENSIONAL BEKK GARCH MODELS”?

http://www.rcfea.org/RePEc/pdf/wp46_12.pdf

or “Lagrangian Dynamical Monte Carlo” ?
http://arxiv.org/abs/1211.3759

I’m afraid the holy grail has not been found. We need a hessian which is local along leapfrog steps, rather than global, we need a hessian which can be efficiently calculated and/or adapted with efficient leapfrogs, and we need to be able jump modes. No published method I have seen satisfies these criteria.

Also of course it has to satisfy detailed balance etc. I tried various methods which did not satisfy detailed balance but gave good local hessians, but the results were (not surprisingly) unreliable.

I have had lengthy discussions with Hoffman and Zhang, and I think we all came to similar conclusions.

My current approach is to use a mix of different methods – NUTS and DREAM and RWM, each has strengths and weaknesses.

I suppose it’s possible I misimplemented the algorithm, though. If you’ve had more success, I’d love to hear about it.