“Schooled”? I had somehow missed the blogosphere discussion of this study of cowpea varieties in Tanzania

The paper was published in AJAE earlier this year.

Abstract of paper:

Randomized controlled trials (RCTs) in the social sciences are typically not double-blind, so participants know they are “treated” and will adjust their behavior accordingly. Such effort responses complicate the assessment of impact. To gauge the potential magnitude of effort responses we implement a conventional RCT and double-blind trial in rural Tanzania, and randomly allocate modern and traditional cowpea seed varieties to a sample of farmers. Effort responses can be quantitatively important—for our case they explain the entire “treatment effect on the treated” as measured in a conventional economic RCT. Specifically, harvests are the same for people who know they received the modern seeds and for people who did not know what type of seeds they got; however, people who knew they had received the traditional seeds did much worse. Importantly, we also find that most of the behavioral response is unobserved by the analyst, or at least not readily captured using coarse, standard controls.

Berk Ozler comments of earlier version:

There is a way to solve this problem: the outcome variable should equal “zero” for anyone who did not harvest cowpeas: that is literally equal to how many KGs they harvested from the seeds provided by the experimenters and it is also the correct intention to treat estimator. Given that I have the attrition rates for each group, I can actually re-calculate the impact sizes in Table 2 by multiplying them with the compliance rate in each group which is equivalent to assigning zeros to all the people who did not harvest cowpeas. That calculation produces 4.6 KGs for MS vs. 3.52 KGs for TS in the open RCT, while the same figures are 4.2 and 3.51, respectively in the double blind RCT: the so called pseudo-placebo effect for those who received traditional seeds is no longer there 3.52 vs. 3.51! Furthermore, the effects for the households that received modern seeds are now further apart 4.6 vs. 4.2 than they are in Table 2, as one would expect if effort is complementary to improved seeds and people who knew that they received MS put in more effort than those who recived the same but did not know it. The story makes a little more sense now than it does in Tables 2 and 3.

via Mind Your Cowpeas and Cues: Inference and External Validity in RCTs | Impact Evaluations.

About mkevane

Economist at Santa Clara University and Director of Friends of African Village Libraries.
This entry was posted in Development thinking. Bookmark the permalink.