Reminder that publication bias is important… but seems not to have greatly affected results in a number of reviews.

I might have added that an important set of explanatory variables might be the “breadth” of the field, the likely incentives for publishing null results, and the “recentness” of the field… in other words, there is likely to be a lot more positive publication bias in a recent, faddish field where only a dozen papers have been published than in a mature, established field where there might be large returns to publishing negative results, in part because the implications of the results have immediate real-world consequences (i.e. as in medicine) rather than unlikely consequences (as in “my DSGE paper will convince the Fed to end quantitative easing in two months rather than 12 months”).

Publication or related biases were common within the sample of meta-analyses assessed. In most cases these biases did not affect the conclusions. Nevertheless, researchers should check routinely whether conclusions of systematic reviews are robust to possible non-random selection mechanisms.

via Empirical assessment of effect of publication bias on meta-analyses | BMJ.