p-values, 2

In this sequel, I want to tell a story which perhaps indicates that the philosophical positions of scientists and statisticians have less influence on what really goes on than much more practical things like publication policies of journals. This story concerns a paper I saw nearly 50 years ago. I hope that this sort of thing couldn’t happen now, but I wish I felt more confident about that.

The paper was in a journal of animal behaviour, I really don’t remember which one but I think it was a “reputable” journal. I haven’t tried to locate it, so everything I say is from memory and thus unreliable. But back then, you needed to have found something statistically significant at the 5% level at least (that is, p ≤ 0.05) to have any hope of getting your work published.

Anyway, the scientist had observed baby birds for the first fifteen days of their life, and had measured some attribute (I don’t recall what) each day. To analyse the results, the value of the attribute on the i-th day was compared with the value on the j-th day, for all pairs {i,j}. One of these pairs of values was found to be significantly different, and on the strength of this significant result, the paper was accepted for publication.

Now I think it hardly needs saying, but apparently it escaped the authors, that the real discovery they had made is that only one pair of results are significantly different, when I would have expected five!

About Peter Cameron

I count all the things that need to be counted.
This entry was posted in exposition and tagged . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.