In response to Andrew Gelman’s extended April Fool’s diatribe on Objections to Bayesian Statistics, Larry Wasserman commented regarding physicists who want  guaranteed frequentist coverage for their confidence intervals that  “Their desire for frequentist coverage seems well justified. Someday, we can count how many of their intervals trapped the true parameter values and assess the coverage. The 95 percent frequentist intervals will live up to their advertised coverage claims. A trail of Bayesian intervals will, in general, not have this property”.

One thing to note about this statement is that it’s just not true.  Confidence intervals produced in actual scientific research are notorious for not covering the true value, even when they are produced using frequentist recipes.  This is why high-energy physicists insist on such absurdly high confidence levels (or absurdly low p-values) before declaring discoveries — what they call “five sigma” evidence, which corresponds to a p-value of less than 10^-6. If taken seriously, quoting such a small p-value would be pointless, since any reader would surely assign a higher probability than that to the possibility that the “discovery” results from fraud or gross incompetence. The high confidence levels demanded are just an ad hoc way of trying to compensate for possible inadequacies in the statistical model used, which can easily make the true coverage probability be much less than advertised (or the true Type I error rate much higher than advertised).

Let’s ignore this, though, since discussions of theory omitting messy practical issues can be valuable.  The next thing to ask, then, is how it is possible that a 90% Bayesian probability interval — which purports to contain the true value with 90% probability —  can contain the true value less than 90% of the time.  A simple example will show how this can happen, and provide insight into whether we should care.

One can trivially get an example of low or zero coverage using a Bayesian method in which some parameter values are assigned low or zero prior probability, but that’s not very interesting, since if you use such a prior, you presumably want such values to be considered low probability (and hence excluded from posterior intervals) even if they’re not especially disfavoured by the data.  But here I’ll give an example in which the coverage is zero for a parameter value that has just as high prior probability as all the other parameter values.

Suppose we have an unknown parameter, θ, with ten possible values, 0, 1, …, 9.  We obtain data, x, which  has nine possible values, 1, …, 9.  If θ=0, the observation x is equally likely to be any of the values 1, …, 9 —  ie, the probabiity of  each of these values is 1/9.  If θ>0, we will observe x=θ with probability one.  Suppose we use a uniform prior for θ, so each of the values 0, 1, … 9 has prior probability 1/10.

Once we observe x, only two values of θ will remain possible — θ=x and θ=0.  By assumption, the ratio of prior probabilities for these two values is one.  The ratio of likelihoods is 1 over 1/9, or 9, in favour of θ=x.  The posterior odds are the product of the prior odds and the likelihood ratio, so we find that the posterior probabilities are 9/10 for θ=x, 1/10 for θ=0, and zero for any other value of θ.  A 90% Bayesian probability region is therefore easy to define — it’s just the set consisting only of the observed value of x.

What’s the coverage of this Bayesian probability region?  Well, if θ>0, the coverage is 100%, since when θ>0 we are guaranteed to observe x=θ.  But when θ=0, the coverage is 0%, since we never produce a posterior probability region that includes 0.  Frequentist coverage is the minimum probability, for any true θ, that the region will include the true θ.  So the coverage for these Bayesian probability regions is zero.

Should zero coverage be cause for worry?  That depends first of all on whether we actually believe the model and the prior that were used to obtain the Bayesian regions, and secondly on whether a posterior probability region actually provides the information that we need.  It’s certainly possible that the answers to both these questions are “yes”, in which case the zero coverage should not be a cause for worry.  But I think that often Bayesian posterior regions are not what we want.

Recall that one way of looking at a 90% frequentist confidence region is as the set of all parameter values for which a hypothesis test would produce a p-value greater than 0.1 — ie, the set of parameter values that are consistent with the data, according to a hypothesis test using a 10% significance level.  From this point of view, the confidence region is a way of telling theorists what theories to discard — namely, all those theories that predict a value for the parameter outside the confidence region.

Does this work for the Bayesian posterior regions in the example above?  If we observe x=9, we construct the 90% posterior region {9}, and conclude that all theories that predict any value for the parameter other than 9 should be discarded (at least if we think 90% is high enough).  This is certainly the right thing to do for theories that predict that the parameter is 1, 2, …, 8, since those values are excluded by the data with certainty.  But what about parameter value 0?  It’s outside the 90% posterior region, but note that its posterior probability of 1/10 is exactly the same as its prior probability.  The observation of x=9 has not reduced the probability that θ=0 at all, so it certainly seems strange to say that θ=0 is now excluded by the data!

I think part of the problem is that reports of experimental results should not be aimed at presenting conclusions, as may seem most natural from a Bayesian viewpoint, but rather at providing the information with which the readers may draw conclusions.  This may be the source of some objections to the prior distribution in Bayesian analysis, which can be seen as corrupting the objective presentation of the experimental results, even though frequentist methods like p-values are not suitable presentations either.  In simple examples like the one above, the experimental results can be communicated fully and objectively (assuming the model is uncontroversial) by reporting the likelihood function — which in this example is 1 for Θ equal to the observed x, 1/9 for Θ equal to 0, and zero for any other Θ.  There is no need for either frequentist confidence regions or Bayesian posterior regions.   Readers can use the likelihood function to produce posterior regions based on whatever priors they like.  (Frequentists using methods that violate the Likelihood Principle are out of luck, but you can’t please everyone.)

In more complex problems with a high-dimensional parameter, however, just presenting the likelihood function is both infeasible and unenlightening.  One solution is to present a marginal likelihood function for just a smaller set of parameters of interest, integrating with respect to a prior distribution for the other “nuisance” parameters.  This only works if the prior for nuisance parameters is relatively uncontroversial, but that may often be the case (or at least, the experimenters may be the best people to formulate this prior).

If you think you must instead produce something like a “Bayesian confidence region”, perhaps the best method is to report the region for which the ratio of posterior probability (or density) to prior probability (or density) is not less than 0.1 (for an analogue of a 90% region).  This can also be thought of as the region containing all  Θ values for which the Bayes Factor is at least 0.1 for a comparison of the model in which this Θ has prior probability one to the model in which Θ has whatever prior you have set up as the “default”.  This can also be seen as the region of Θ where a theorist who assigns prior probability 1/2 to their pet theory predicting that value of Θ would not abandon their theory (using a 90% confidence level) if the they use your “default” prior for parameter values conditional on their theory being false.  For the example above, the regions produced in this way (with uniform default prior) would consist of the observed x plus 0.  Coverage would be 100%.

I hasten to add that this suggestion assumes a context in which various theories predict specific values for the parameter, and our interest is in which of these theories is true.  This is quite unlike many applications of confidence intervals, in which the parameters are things such as the average daily caloric intake of Canadian adults, or the regression coefficient of income on years of education.  For such parameters, the likelihood function (or marginal likelihood function) should be reported.  If this likelihood happens to be approximately normal, and concentrated in a region where any reasonable prior would be nearly uniform, you could go ahead and present a 90% posterior region if you’re so inclined.

Click on image for larger version.

Downtown Toronto, August 2008. Mamiya C220 TLR, 80mm 1:2.8 lens, Fuji Reala 100 film, Nikon Coolscan 9000.

I’ve been busy with teaching, so I’m only now getting around to posting the answers to the second applied statistic comprehensive exam question that I posted (here).

Originally, I’d thought that posting the answer would be simply a matter of extracting the answer I’d already written before.  But looking it over before posting, I noticed that I’d made an additional unintentional error in the analysis that the question asks you to critique! Of course, one could say that the more errors the better, but I did need to update my answer to reflect this.

The error is that in the first analysis presented, I had intended to include interaction terms between treatment and covariates such as sex in the regression shown.  Due to some sort of momentary brain failure, I instead just put these covariates in by themselves.  When writing up my answer later (which students who wrote the exam got), my mind was set on the idea that I’d put in interaction terms, so this error wasn’t reflected in that answer.  I don’t think this had any significant effect on the marking, fortunately, since the comments on the later analyses aren’t really affected.  It does show how easy it is to keep seeing what you expect to see.

Here’s the PDF file with the answers.  The questions are included as well, so no need to refer back to them.

Unlike the two design flaws I posted about before (here, here, and also here), where one could at least see a reason for the design decision, even if it was unwise, this design flaw is just  incomprehensible.  For no reason at all that I can see, R allows one to use zero as a subscript without triggering an error.  (Remember that in R, indexes for vectors and matrices start at one, not zero.)

This is of course a terrible decision, because it makes debugging harder, and makes it more likely that bugs will exist that have never been noticed.

So what does R do with a zero subscript, seeing as it’s meaningless?  It just ignores it, which is possible because it views all numeric subscripts as vectors, that extract or replace a set of elements, not necessarily just one.   So R simply removes all zeros from a vector used as a subscript, producing a shorter vector.

Here’s what happens (with the current version of R, 2.7.2):

   > a
   [1] 10 20 30 40 50
   > a[0]
   numeric(0)
   > a[c(4,2)]
   [1] 40 20
   > a[c(4,0,2,0)]
   [1] 40 20
   > a[0] <- 7
   [1] 10 20 30 40 50
   > a[c(4,0,2,0)] <- 7
   [1] 10  7 30  7 50

Contrast this with what happens when you use a subscript that is too large:

   > a
   [1] 10 20 30 40 50
   > a[7]
   [1] NA
   > a[c(4,7,2)]
   [1] 40 NA 20
   > a[7] <- 7
   [1] 10 20 30 40 50 NA  7

Extending vectors automatically when an assignment is made beyond the end can obviously be useful (though it might be wiser not to).  Returning NA when extracting an element beyond the end is also a sensible action (though signalling an error immediately might be more useful for debugging). And negative subscripts are usefully defined as referring to their complement. But what possible use is there for ignoring zero subscripts rather than signalling an error?

It’s perhaps belabouring the obvious, but let me explain that signalling an error when a zero subscript is used is desirable because this is a very common sort of program bug. It can easily arise when a program is scanning backwards through the vector elements, and goes one step too far. It can also easily arise when data is initialized to zeros, with the intent to replace the zeros with something sensible later, but actually some zeros are never replaced. The way R behaves when zero is used as a subscript when replacing elements is particularly bad, since doing nothing at all can easily lead to an apparently working program that produces wrong answers.  (The behaviour of returning an empty vector when zero is used as a subscript when extracting an element is more likely to produce an error later on, so that at least the problem will be evident.)

So what should be done?  That’s easy — change R so that use of zero as a subscript produces an immediate error.  That’s trivial to do (mixing positive and negative subscripts produces an immediate error now, so the apparatus for it must be there).  Might that break some existing programs?  Yes, it will.  But 99.9% of those programs are already broken.  The users just don’t know it, thinking that the answers they get are correct when they’re not.  The remaining 0.1% of these broken programs were written by really stupid programmers who thought that exploiting an obscure and unwise feature in order to produce a really hard-to-understand program was a good idea.  It wasn’t.

Along with this, R should be changed so that using NA as a subscript when replacing elements in a vector also produces an error.  What to do with NA subscripts used to extract elements is a little bit harder to decide, but it seems to me that something about the following is a bit funny:

   > a
   [1] 10 20 30 40 50
   > a[NA]
   [1] NA NA NA NA NA
   > a[NA+0]
   [1] NA

PhD students in the Dept. of Statistics at the University of Toronto normally write three comprehensive exams at the end of their first year, in Probability, Theoretical Statistics, and Applied Statistics. Below is a question I set for the 2007 exam in Applied Statistics. It may be an interesting exercise for others too. It should in theory be doable by someone with just a good introductory undergraduate course in statistics, including multiple regression. However, many PhD students had difficulty with it, so I wouldn’t say it’s easy.

The question is here. I’ll post my answer in a week or so.

My previous post with a question from the 2008 exam is here.

Update: Here is the post with the answers.

Click on image for larger version.

Nikon FG, Nikkor AIS 1:1.4 50mm, Black’s ISO 200 film, Nikon Coolscan V.

I have a wonderful 11-month-old daughter, who thankfully is entirely healthy. During the pregnancy, my wife and I were of course worried about the possibility of a congenital defect, of which the most prominent is Down Syndrome. Today, couples must make a series of complex decisions — whether to have a screening test for Down Syndrome, whether (based on its result) to have a more risky diagnostic test, and of course, what to do if the final result is that the fetus has Down Syndrome. These decisions depend on moral judgements, on various facts regarding the nature of the fetus at various ages, regarding the nature of Down Syndrome, and regarding the reliability and dangers of the tests, and finally, on the proper way to use this information to make a decision.

This last aspect is in the domain of decision theory, and will be the main focus of this post. Decision theory purports to show how a decision-maker should use the probabilities of the various possible outcomes along with their personal “utilities” for these outcomes to make a rational decision, which maximizes their expected utility. The validity of decision theory as a guide to rational action has often been challenged. The Allais Paradox describes one situation where decision theory does not accord with the judgements of many people, and some argue that the fault is not with these people, but rather with decision theory. Interestingly, Down Syndrome testing involves an analogue of the Allais Paradox.

For readers who aren’t familiar with pre-natal tests for Down Syndrome, I’ll first present the main facts:

• Down syndrome results from a chromosomal abnormality, which causes cognitive impairment and a wide range of physical problems. Both the degree of cognitive impairment and the severity of the physical problems are highly variable.
• The risk of Down Syndrome increases with maternal age, from less than 1 i 1000 for young women, to 1 in 100 for women age 40, to 1 in 30 for women age 45, and to 1 in 12 for the rare birth to a woman age 49.
• Whether a fetus has Down Syndrome can be determined to high accuracy between 16 and 22 weeks into the pregnancy by performing amniocentesis, which is an invasive procedure that has about a 1 in 200 chance of causing the fetus to miscarry. Results are available after 2 to 4 weeks.
• Several non-invasive screening procedures (blood tests and ultrasound examination) that are performed at various times from 11 weeks to 16 weeks into the pregnancy can provide information on how likely the fetus is to have Down Syndrome. Combined with information on maternal age, these can give a probability that the fetus has Down Syndrome.
• The results of these screening tests never definitely indicate Down Syndrome. They are used only to decide whether amniocentesis should be done.

Pretty much the only point of testing for Down Syndrome is to provide the option of terminating the pregnancy if the fetus is confirmed to have Down Syndrome. (The only other purpose I can see would be to prepare mentally or financially for having a child with Down Syndrome, but given the risk of amniocentesis, I think few well-informed couples would regard this as a sensible reason to have it done.) The tests available in Ontario are described at this web site, which gives all the facts above (plus more, I’ve slightly simplified the available options), but which strangely makes no mention of pregnancy termination, except buried in the PDF files of some pamphlets. This page by the American College of Obstetricians and Gynecologists also manages to avoid any mention of pregnancy termination. One has to wonder what the rationale might be for this stunning failure to inform people of the purpose of the tests they discuss.

Clearly, if you are certain that you wouldn’t want to terminate the pregnancy even if you were sure the fetus has Down Syndrome, then you shouldn’t do any of these tests. At least, that’s so if we ignore the complication that some of these tests can also detect some rarer conditions, such as Trisomy 18, which are much more severe than Down Syndrome, as well as some rarer conditions that might be treatable.

I suspect that few people are so certain about what they would actually do if faced with a decision of whether to terminate a Down Syndrome pregnancy. On the question of when a fetus becomes “human” (possessing human rights), most people are not extremists — they neither believe that a fertilized egg is fully human, nor believe that a fetus just about to be born has no moral standing. Instead, most people are unsure when a fetus becomes human, and think in any case that the process is a gradual one. Furthermore, where their views fall within this non-extreme range may well be altered by the experience of pregnancy (especially the first time). Also, on learning that their unborn child has Down Syndrome, most couples are likely to learn more about Down Syndrome before making a decision. I do assume that few people regard Down Syndrome as a desirable condition, and indeed, that most people would consider it unethical to conceive a child if they knew for certain (before conception) that the child would have Down Syndrome. (Note that this is a hypothetical situation which never arises in practice.)

Most people should therefore have the non-invasive tests, and then think about the issue more once they have the results. The first question is whether to have amniocentesis done, which would provide an accurate diagnosis of whether the fetus has Down Syndrome, but which has a 1 in 200 chance of causing a miscarriage.

It’s at this point that decision theory has something to say. Suppose that the non-invasive tests (along with maternal age) give a 1 in 1000 chance of Down Syndrome. You could try to directly weigh the benefit (given this chance) of using amniocentesis to confirm whether the fetus really has Down Syndrome against the 1 in 200 chance that amniocentesis would cause a miscarriage, and come to a decision. But applying the “Independence Axiom” of decision theory may clarify the situation.

Let’s ignore the 1 in 200,000 chance that the fetus has Down Syndrome and would be lost to a miscarriage if amniocentesis were done. We can then visualize the decision in terms of 1000 hypothetical pregnancies. In one of these pregnancies, but not the others, the fetus has Down Syndrome. In 5 of these pregnancies, a miscarriage will occur if amniocentesis is done. In 994 of these pregnancies, the fetus does not have Down Syndrome, and amniocentesis would not cause a miscarriage. For these last 994 pregnancies, it makes no difference whether amniocentesis is done or not. These pregnancies can therefore be ignored when making a decision.

Pretending, therefore, that one of the remaining 6 pregnancies is the real one, the decision looks like this: If you do nothing, there’s a 1 in 6 chance you’ll have a child with Down Syndrome. If you have amniocentesis done, that’s effectively like terminating the pregnancy, since for 5 of the 6 pregnancies miscarriage will result, and for the other (with Down Syndrome) you would likely decide to terminate the pregnancy. The decision whether or not to have amniocentesis when there’s a 1 in 1000 chance of Down Syndrome is therefore equivalent to a hypothetical decision whether or not to terminate a pregnancy when there is a 1 in 6 chance of Down Syndrome and no further diagnostic test is available. Of course, if the non-invasive tests produced a different probability, we’d have a different equivalent problem — for example, with a 1 in 2000 probability of Down Syndrome, the equivalent decision is whether to terminate a pregnancy when there’s a 1 in 11 chance of Down Syndrome.

Is making a hypothetical equivalent decision of this sort any easier than making the original decision? I think so, because the probabilities are less extreme, and the decision is more similar to other decisions that you may have already considered.

One highly relevant comparison is that about 1 in 30 children are born with some sort of serious congenital defect, most of which can’t be diagnosed before birth. If you don’t find this level of risk acceptable, you shouldn’t be thinking of having children. The chance of Down Syndrome in the hypothetical decision described above will often be close enough to this that you can think about the risk in the same way, and decide whether you regard it as also being acceptable.

A few years ago, before any prenatal tests were available, a pregnant woman age 49 would have faced the choice of whether to terminate her pregnancy based simply on the overall risk of 1 in 12 of Down Syndrome at her age. This is about equivalent to the choice of whether to have amniocentesis done when the non-invasive tests (plus age) give a 1 in 2000 probability of Down Syndrome.

It may also be helpful to compare with the current world-wide infant mortality rate (deaths in the first year) of around 1 in 20. Infant mortality is around 1 in 200 in developed countries, and around 1 in 6 in a few countries (such as Angola and Afghanistan). Infant mortality in the United States in 1950 was about 1 in 30. These figures tell you something about what other people have regarded as acceptable risks.

Ultimately, of course, your decision will still depend on your personal risk tolerance, your view of Down Syndrome, your view of the humanity of a fetus age 16 weeks or more, and the probability of another pregnancy if you terminate this one.

This last point would seem to be the dominant consideration for anyone who does not view a 16-week (or possibly up to 24-week) fetus as human. Its implication is that older women who want a child should be less inclined to have amniocentesis done than younger women, for a given probability of carrying a Down Syndrome fetus. (Of course, maternal age is one factor on which this probability is based.) This recent post (which I found via Marginal Revolution) discusses this issue.

The equivalence I present above depends, of course, on the validity of the Independence Axiom. In my view, it is obviously correct, even if its implications are not all obvious. The situation with Down Syndrome testing is very analogous to that in the Allais Paradox, which has been taken by some as a refutation of the Independence Axiom, though such a conclusion seems unjustified to me. I think the Down Syndrome testing situation is more interesting, and more real, than the gambling scenario in the typical presentation of the Allais Paradox. (Down Syndrome and decision theory has also been discussed in a comment by A. P. Dawid in Statistical Science, November 1986.)

You will have to judge for yourself whether your intuition satisfies the Independence Axiom, or whether the decision you would intuitively favour for the original problem — whether to have amniocentesis with a 1 in 200 chance of miscarriage if there’s a 1 in 1000 chance of Down Syndrome — differs from the decision you would intuitively favour in the “equivalent” problem — whether to terminate a pregnancy due to a 1 in 6 chance of Down Syndrome.

Suppose that you do find a difference in your intuitions. Those who argue against decision theory (and the Independence Axiom in particular) assume that such differences cast doubt on its validity. But what would be the purpose of developing a theory of decision making if everything it told you was intuitively clear to you anyway?

It’s actually a great benefit of decision theory that it demonstrates that problems which seem different to some people are actually equivalent. This provides an opportunity for further thought, leading to a more satisfactory decision. Analogous situations arise with probability theory, where inconsistent subjective probability judgements indicate that more thought is needed. One should note, however, that sometimes the resolution of such inconsistencies is not that one or both judgements are wrong, but that both are right, with the apparent conflict being due to a failure when formalizing the problem to include some non-obvious but relevant aspects (such as the effect of the decision made on the decision-maker’s reputation for good decision-making).

Some recent studies have shown much lower risk of miscarriage from amniocentesis than 1 in 200 (1 in 1000 or lower), but caution in interpreting these results is needed, since they may not apply to the particular facility where you would have amniocentesis done. Better non-invasive tests, such as ones that look at fetal cells in the mother’s blood, may also become available. We can hope that this interesting decision theory problem will soon cease to be real.

I’ve previously posted about two design flaws in R. The first post was about how R produces reversed sequences from a:b when a>b, with bad consequences in “for” statements (and elsewhere). The second post was about how R by default drops dimensions in expressions like M[i:j,] when i:j is a sequence only one long (ie, when i equals j).

In both posts, I suggested ways of extending R to try to solve these problems. I now think there is a better way, however, which solves both problems with one simple extension to R. This extension would also make R programs run faster and use less memory.

Recall the problems, and the solutions I proposed previously… To stop sequences from reversing, we need a new operator to use rather than 1:n, which suffers from the reversal problem when n is zero (which can’t be changed for compatibility reasons). I suggested 1:>:n. To stop dimenions from being dropped, I suggested using semicolons rather than commas to separate subscripts. My new suggestion is that in the most common case where the indexing vector is a sequence, we could use the same new operator introduced to solve the reversing problem — ie, we write M[1:>:n,] to get the first n rows, and define it so that the result isn’t converted to a vector when n is one.

Now, this may seem like it won’t work. If 1:>:n returns a vector, then when n is one, it’s a vector of length one, which has to (by default) lead to the dimension being dropped if ordinary subscripting by scalars is to work (since scalars in R are really vectors of length one).

The solution is for 1:>:n to not return a vector, but instead return a new data type that just records its two operands. This new data type — perhaps it could be called an indexing pair — would have to be recognized by “for” statements and by the subscripting operator. A dimension indexed by such an indexing pair would never be dropped. One could add an operator :<: as well, for iterating or subscripting in descending sequence, though this is much less common. If this descending operator is omitted, I think .. (two periods) would be a better name than :>: for the ascending indexing pair operator (but it unfortunately has no obviously mnemonic descending counterpart).

One disadvantage of this solution is that it doesn’t address the dropped dimension problem in the general case where the vector index may not be a sequence. But indexing by an ascending sequence is by far the most common case, so having to write drop=FALSE for the others may be OK. (However, perhaps knowledge of the drop=FALSE option would be less common if it is needed less often.) Similary, this solution doesn’t address the problem of reversing sequences outside the context of for loops and subscripting, but those are the most common uses.

One extra advantage of introducing indexing pairs is that they will take up a trivial amount of storage. In contrast, if you use 1:1000000 to iterate a million times in a for loop, R will allocate 4 Megabyes of storage to hold this sequence. Producing this sequence also takes time, of course. It would be possible for R to avoid this cost when using 1:1000000 in a for loop, by treating this combination specially, but it doesn’t (in version 2.4.1 at least):

   > gc()
            used (Mb) gc trigger (Mb) max used (Mb)
   Ncells 234834  6.3     467875 12.5   407500 10.9
   Vcells 104319  0.8     786432  6.0   690698  5.3
   > for (i in 1:1000000) if (i==1000000) print(gc())
            used (Mb) gc trigger (Mb) max used (Mb)
   Ncells 234855  6.3     467875 12.5   467875 12.5
   Vcells 604321  4.7     905753  7.0   720903  5.5
   > gc()
            used (Mb) gc trigger (Mb) max used (Mb)
   Ncells 234904  6.3     467875 12.5   467875 12.5
   Vcells 104328  0.8     786432  6.0   720903  5.5

Notice how the memory usage goes up by 3.9 Megabytes during the for loop.

Another extra advantage of introducing :>: (or ..) is that the precedence of this operator could be made lower than that of any other operator (with no loss, since an indexing pair wouldn’t be a valid operand for any other operator). Expressions like 1:>:n-1 would then do what the programmer meant (unlike 1:n-1).

So, does anyone see any flaws in this solution?

This post links to a Question I set for an applied statistics PhD comprehensive exam. My answers for this question are here (the question is repeated there, so no need to look at the old post).

Note that my answers are more elaborate than I would expect a student to write on an exam. I also gave credit for discussions that showed some insight, even if the final answer wasn’t completely correct.

Click on image for larger version.

Nikon FG, Nikon Series E 1:1.8 50mm, Black’s ISO 200 film, Nikon Coolscan V.