The other day on “On the Media” with Brooke Gladstone, I hear while driving up to San Francisco, a story from Radiolab about positive publication bias and regression to the mean.  The story involves a study done some years ago that found that people who were asked to write down a detailed description of a bank robbery seen on video, right after watching the video, performed worse, not better ,at recall than a control group that was not prompted to remember details of the bank robber.  Attempts to replicate found the measured difference fading away.

Regression to the mean?  Positive publication bias?

No, the piece emphasized something else.  The finding of a difference between treatment and control gradually faded towards zero, rather than bouncing around.  The author speculated… perhaps the “ether of reality” was affected by scientific observation (by psychologists… and others too?).  Human interaction and minds keep changing when scientists observe stuff even if the minds are not aware of the observations.

Preposterous!  But then I realized.  Hmmm.  The interviewer never asked how many replication studies had been done.  Sounded like three or four.  The interviewer never asked whether any other replication studies had “smooth” regression to the mean rather than a bounce around.  The interviewer never asked any of a dozen other obvious questions.  And THEN I realized… it was all tongue in cheek… because they commented on how the original study was so compelling that it would be talked about at “dinner parties,” and now here was another finding so compelling that it also would be talked about at dinner parties.

So NPR kindly provided me with a non-study finding that I could talk about that evening.  For free.