Trying to think what you might be thinking, however, my guess is that you think that all real data is discrete, not continuous, and hence all real likelihoods are based on probability mass functions, not probability density functions. There’s something to be said for this view, and it does formally eliminate the inconsistency of the MLE in this example if you assume that the data has limited precision.

However, in high-dimensional problems the finite space of possible data sets is extremely large, even assuming individual values are rounded to not-too-much precision. It may then be more enlightening to consider continuous data (even if that’s an unrealizable idealization) than to trust that the MLE is guaranteed to be consistent in finite settings, when convergence to the correct value may in practice occur extremely slowly.

]]>L(Data)=P(Data|Model). For normally distributed IID data, this likelihood represented by the product point sampled Gaussians CAN be a good approximation of un-normalized probability, but aren’t always. Here they are not, for the peaked distribution surrounding the point closest to zero (singling out that point as sigma ->0) .

As likelihood is a probability, L(D) is never >1. I believe the issue here is in a problematic estimate of likelihood, not inconsistency of the MLE.

]]>Regarding the choice of step-size parameter for NUTS, specifically at this point that “the sampler would at some iteration move to the region that needs a small step-size, and then get stuck there, as all the proposals are rejected — a problem that is easily diagnosed. Unfortunately, that’s not what will happen. Since the sampler leaves the correct distribution invariant, if it would be stuck for a very long time in some region, it must also be very unlikely to enter that region.”

Is it applicable to conventional HMC only, since there is no eventual rejection in NUTS since it is essentially a slice sampler?

]]>I am totally for the implementation of correct generalizations of operators (such as the v[-ix] that you mention in your presentation when ix has length 0).

I wonder why the R Core team is so uninterested in these improvements…

Also was happy to learn the origin of the pqR name… I thought the name was *purely* based on the alphabetical order of the letters involved :-)

]]>I plan to implement a number of other language extensions, at which point I’ll be better able to see what possible implementation or compatibility problems there are for the whole set. That might be a better time to make more formal proposals.

You can see some of these plans (which may not exactly match what I eventually implement) at http://www.cs.utoronto.ca/~radford/ftp/pqR-Rusers.pdf

]]>http://www.r-bloggers.com/get-involved-with-the-r-consortium/ ]]>