Realist evaluation and randomised controlled trials for testing program theory in complex social systems

Scientific evaluation seeks to develop and test theories that describe and explain the value of interventions into the world. Realist approaches to scientific evaluation tend to be strong on theory and explanation, but lack adequate tests or means of validating theory. The focus of this article is the potential for randomisation and experimentation to provide evidence for transfactual (i.e. reusable or portable) context–mechanism–outcome configurations (CMOs) in complex adaptive systems. The article is not concerned with attribution of outcomes to past programs but for developing scientific knowledge that can be used for future interventions. It seeks to elucidate the warrant that underpins the randomised controlled trial (RCT) and why it is useful in some fields of science but less so in complex social systems. Realist RCTs are considered but rejected; instead a form of propensity score matching is proposed for testing realist program theory, estimating the effect size of a purported CMO, and generating scientific knowledge for developing more effective interventions into complex social systems.