Cono Ariti, Senior Research Analyst at the Nuffield Trust, then described the retrospective matched control design used by the Nuffield Trust in conducting quantitative evaluation.
Cono first set out the problems associated with carrying out evaluation of health models and interventions in practice. He highlighted that while we know RCTs are the ‘gold standard’ to test effectiveness of an intervention, often this method is not feasible or ethical, and the generalisability is limited. However, if an observational study is undertaken, typically there is no natural experiment that exists and often no comparable group. Cono also explained that evaluation is often only thought of after an intervention has been implemented. These issues have led to the development of other, alternative and often innovative methods of evaluation.
A retrospective matched control design uses routine databases to match individuals who have received an intervention with a control. The advantages of this method are that it:
- makes it possible to plan retrospectively
- presents a robust method in terms of generating good quality evidence
- reflects the real-world effect of an intervention rather than testing effectiveness
- is cheaper than a traditional RCT.
To undertake this method locally, Cono set out some key questions to consider in the slide above.
The conference audience asked about how evaluators in local areas could be encouraged and trained to apply these methods more widely. The Nuffield Trust has written a concise guide on the steps that need to be undertaken to use this method of evaluation. Many of the initial steps can be carried out locally with minimal external support, but it may then be necessary to draw on statistical expertise to carry out the specific analytical methods.