Innumeracy at the Globe and Mail

In the June 23 print edition of the Globe and Mail (billed as “Canada’s National Newspaper”), there’s an article on data centres (“Hewers of wood, storers of data”), in which, on page B4, one can read the following:

Greenpeace recently released a report that said if the Internet were a country, it would be the fifth-largest consumer of energy, largely because of the massive data centres that run unseen in the background. The group estimated that the centres will use 1.9 billion kilowatt hours of electricity by 2020 — more than the amount currently used by Canada, France, Germany and Brazil combined. (The average US home uses 8,000 kilowatt hours a year.)

An exercise for the reader: How many logical fallacies, arithmetic errors, or contradictions of common knowledge can you find in this passage?

I haven’t tried to determine whether these fallacies originate in the (unidentified) Greenpeace report, or are original to the Globe and Mail.

Two textbooks on probability using R

This fall, I’ll be teaching a second-year course on Probability with Computer Applications, which is required for Computer Science majors.  I’ve taught this before, but that was five years ago, so I’ve been looking to see what new textbooks would be suitable.  The course aims not just to use computer science applications as examples, but also to reinforce concepts of probability with programs, and to show how simulation can be used to solve problems that aren’t easily solved analytically. I’ve used R for the programming part, and plan to again, so I was naturally interested in two recent textbooks that seemed to have similar aims:

Introduction to Probability with R, Kenneth Baclawski, Chapman & Hall / CRC.

Probability with R: An Introduction with Computer Science Applications, Jane M. Horgan, Wiley.

I’ve now had a look at both of these textbooks.  Unfortunately, they are both seriously flawed.  Even more unfortunately, although some of the flaws in these books are particularly striking, I’ve seen similar, if usually less serious, problems in many other textbooks.

Does coverage matter?

In response to Andrew Gelman’s extended April Fool’s diatribe on Objections to Bayesian Statistics, Larry Wasserman commented regarding physicists who want  guaranteed frequentist coverage for their confidence intervals that  “Their desire for frequentist coverage seems well justified. Someday, we can count how many of their intervals trapped the true parameter values and assess the coverage. The 95 percent frequentist intervals will live up to their advertised coverage claims. A trail of Bayesian intervals will, in general, not have this property”.

One thing to note about this statement is that it’s just not true.  Confidence intervals produced in actual scientific research are notorious for not covering the true value, even when they are produced using frequentist recipes.  This is why high-energy physicists insist on such absurdly high confidence levels (or absurdly low p-values) before declaring discoveries — what they call “five sigma” evidence, which corresponds to a p-value of less than 10^-6. If taken seriously, quoting such a small p-value would be pointless, since any reader would surely assign a higher probability than that to the possibility that the “discovery” results from fraud or gross incompetence. The high confidence levels demanded are just an ad hoc way of trying to compensate for possible inadequacies in the statistical model used, which can easily make the true coverage probability be much less than advertised (or the true Type I error rate much higher than advertised).

Let’s ignore this, though, since discussions of theory omitting messy practical issues can be valuable.  The next thing to ask, then, is how it is possible that a 90% Bayesian probability interval — which purports to contain the true value with 90% probability —  can contain the true value less than 90% of the time.  A simple example will show how this can happen, and provide insight into whether we should care.

Answers to Applied PhD Comprehensive Question #2

I’ve been busy with teaching, so I’m only now getting around to posting the answers to the second applied statistic comprehensive exam question that I posted (here).

Originally, I’d thought that posting the answer would be simply a matter of extracting the answer I’d already written before.  But looking it over before posting, I noticed that I’d made an additional unintentional error in the analysis that the question asks you to critique! Of course, one could say that the more errors the better, but I did need to update my answer to reflect this.

The error is that in the first analysis presented, I had intended to include interaction terms between treatment and covariates such as sex in the regression shown.  Due to some sort of momentary brain failure, I instead just put these covariates in by themselves.  When writing up my answer later (which students who wrote the exam got), my mind was set on the idea that I’d put in interaction terms, so this error wasn’t reflected in that answer.  I don’t think this had any significant effect on the marking, fortunately, since the comments on the later analyses aren’t really affected.  It does show how easy it is to keep seeing what you expect to see.

Here’s the PDF file with the answers.  The questions are included as well, so no need to refer back to them.

Applied Statistics PhD Comprehensive Question #2

PhD students in the Dept. of Statistics at the University of Toronto normally write three comprehensive exams at the end of their first year, in Probability, Theoretical Statistics, and Applied Statistics. Below is a question I set for the 2007 exam in Applied Statistics. It may be an interesting exercise for others too. It should in theory be doable by someone with just a good introductory undergraduate course in statistics, including multiple regression. However, many PhD students had difficulty with it, so I wouldn’t say it’s easy.

The question is here. I’ll post my answer in a week or so.

My previous post with a question from the 2008 exam is here.

Update: Here is the post with the answers.

Down Syndrome and Decision Theory

I have a wonderful 11-month-old daughter, who thankfully is entirely healthy. During the pregnancy, my wife and I were of course worried about the possibility of a congenital defect, of which the most prominent is Down Syndrome. Today, couples must make a series of complex decisions — whether to have a screening test for Down Syndrome, whether (based on its result) to have a more risky diagnostic test, and of course, what to do if the final result is that the fetus has Down Syndrome. These decisions depend on moral judgements, on various facts regarding the nature of the fetus at various ages, regarding the nature of Down Syndrome, and regarding the reliability and dangers of the tests, and finally, on the proper way to use this information to make a decision.

This last aspect is in the domain of decision theory, and will be the main focus of this post. Decision theory purports to show how a decision-maker should use the probabilities of the various possible outcomes along with their personal “utilities” for these outcomes to make a rational decision, which maximizes their expected utility. The validity of decision theory as a guide to rational action has often been challenged. The Allais Paradox describes one situation where decision theory does not accord with the judgements of many people, and some argue that the fault is not with these people, but rather with decision theory. Interestingly, Down Syndrome testing involves an analogue of the Allais Paradox.

Answers to Applied PhD Comprehensive Question #1

This post links to a Question I set for an applied statistics PhD comprehensive exam. My answers for this question are here (the question is repeated there, so no need to look at the old post).

Note that my answers are more elaborate than I would expect a student to write on an exam. I also gave credit for discussions that showed some insight, even if the final answer wasn’t completely correct.

Applied Statistics PhD Comprehensive Question #1

PhD students in the Dept. of Statistics at the University of Toronto normally write three comprehensive exams at the end of their first year, in Probability, Theoretical Statistics, and Applied Statistics. Below is a question I set for the 2008 exam in Applied Statistics. It may be an interesting exercise for others too. It should in theory be doable by someone with just a good introductory undergraduate course in statistics, including multiple regression. However, many PhD students had difficulty with it, so I wouldn’t say it’s easy.

The question is here. I’ll post my answer in a week or so. I’ll also post a question from 2007 sometime.

Update: Here is the post with the answers.