Asymmetrical limits for the zone of equivalence (Chapter 21)

Tomasz Kowalczyk from Copernicus Therapeutics, Inc. kindly pointed out:

"Page 151, third paragraph. The argument given in this paragraph still does not explain why the zone of equivalence is not symmetrical around 100%. It is true that the zones of equivalence are the same for the test/reference ratio and for the reference/test ratio (with inverted limits), but this fact (although convenient) does not explain why each of these two zones is asymmetrical. The zone is asymmetrical, because it is supposed to contain an asymmetrical confidence interval. The 90% CI is asymmetrical, because this is the CI of the geometric mean ratio (or ratio of geometric means) of peak drug concentrations (Cmax). (On page 95 you state: “Note that the CI of the geometric mean is not symmetrical around the geometric mean”.) The mean is geometric, rather than arithmetic, because the pharmacokinetic measures AUC and Cmax are statistically analyzed after log transformation, as recommended by the FDA. The primary outcome of the analysis is thus the 90% CI for the difference in the arithmetic means of the log transformed data, which is then back-converted to the 90% CI for the ratio of the geometric means. The 90% CI for the difference in the means of log transformed data is perfectly symmetrical (around the mean difference), so is the zone of equivalence if expressed as logarithms: log100+/- log1.25, if the logarithms to the base 10 are used and the ratio is expressed as percentage. The zone of equivalence for the ratio is also perfectly symmetrical but in the multiplicative (not only practical) sense: 100*1.25=125 for the upper limit, and 100/1.25=80 for the lower limit.  

For this reason, Figures 21.2 & 21.3  are incorrect. Since the 80-125% zone of equivalence applies to geometric means (see the previous comment), the CIs should be asymmetrical around the means. The figures, as printed, show symmetrical confidence intervals."

Dangers of overfitting (Chapter 38)

Many studies have purported to predict whether couples will divorce. But these are all done by fitting a complicated model with many independent variables, without validating the model with new data. Heyman and Selp demonstrate the need to validate models.

They used stepwise logistic regression to develop a model to predict divorce in one group of subjects. They had lots of variables (and interactions) so were able to find a model that worked quite well. When predicting a couple would divorce, the equation was correct 65% of the time (positive predictive value). When predicting that a couple would not divorce, the equation was correct 98% of the time (negative predicted value). Sounds useful.

They then applied that same model to a new group of subjects. Not surprisingly, the predictions don't work very well when used with new data.  When predicting a couple would divorce, the model was correct only 29% of the time. Not very impressive, considering that 33% of their subjects had divorced.

This makes an interesting data showing the dangers of overfitting, and the need to test models on new data sets.

 Heyman, R.E,  and Smith-Slep, A.M. The Hazards of Predicting Divorce Without Crossvalidation. J Marriage Fam (2001) vol. 63 (2) pp. 473-479

This same point is made, in plainer language, in an article in Slate.  

Regression to the mean (Chapter 1 and 33)

Matt Briggs reviews, in depth, an uncontrolled study of the effectiveness of Yoga and points out the paper really doesn't show more than regression to the mean. It is a fun read, and makes important points about how to read clinical papers.

Statistics Case Study

One consequence of heart failure is that the heart gets larger (cardiomegaly). This is a physiological adaptation to allow the heart to pump enough blood to perfuse the organs. Eventually the heart grows so large it is not able to pump blood efficiently. Therefore it makes sense to seek a drug that would reduce cardiomegaly in heart failure.

This study tested such a drug in an animal model of heart failure. Half the rats were given the surgery to create heart failure, and half were given sham surgery. In each of those groups, half of the rats were injected with an experimental drug and half were injected with vehicle as a control. (The data are real, but the investigators prefer to remain anonymous).

	Sham surgery			CHF
	Mean	SD	n	Mean	SD	n
Vehicle	0.3124	0.0211	10	0.5481	0.0723	9
Drug	0.3518	0.0251	10	0.4669	0.0768	10

Is the drug effective in blunting the increase in heart weight?

Think about how you would analyze the data before reading the case-study. 

When you have thought about how you would analyze the data, read this 15 page case-study. Here is the corresponding Prism file. Please email me with your suggestions about this case study, or ideas for future case studies.

Is the distribution of bombs random (Chapters 1, 6 and 26)?

Each circle represents the spot where a rocket sent from Germany landed in London. Is there a pattern? It sure looks like it, but in fact the distribution is random, matching the distribution of the Poisson distribution. A chi-square test compares the expected and observed distributions  (RD Clarke, Journal of the Institute of Actuaries, vol. 72, 1946, p. 481). Tierny discusses psychological aspects of this example -- people are more likely to see patterns in random data when very stressed. 

The twelve most important concepts in statistics.

"If you know twelve concepts about a given topic you will look like an expert to people who only know two or three."   Scott Adams, creator of Dilbert

Here is my attempt at explaining the twelve key concepts in statistics.

Do beautiful people have more daughters? (Multiple comparisons; Chapter 23)

It has been published that beautiful people tend to have more daughters than sons. Andrew Gelman reviews these data in a very general article that really is about statistical thinking, and how easy it is to be mislead by statistics. The conclusion does not seem to be solid. 

Reference: American Scientist, 97:310-316, 2009

Coin flipping: Not entirely random.

It turns out that flipping a coin is not always random!

Mammograms: Sensitivity, specificity, etc. (Chapter 42).

The US Preventive Services Task Force has published new guidelines on use of mammograms to detect breast cancer, recommending that they begin at age 50 (rather than 40). Here is a summary in the New York Times, and comments by Matt Briggs (and more). Use this web calculator to play with the numbers. This is a good example to review the concepts of sensitivity and specificity, and false-positive and false-negative test results. This article, written by Dr. Daniel Frank for his patients, does a great job of explaining the big-picture and the concept of Number Needed to Treat (although he never actually uses that term).