Challenges in the impact evaluation of behaviour change interventions: The case of sea turtle meat and eggs in São Tomé

Robust impact evaluations are needed for conservation to learn and grow as a field. Currently we lack a large body of evidence on the effects of behaviour change interventions in social‐ecological systems. By uncovering mechanistic relationships and establishing causality we can refine future programmes to enhance likelihood of effectiveness. Although a range of sophisticated methodological approaches to evaluation have been developed, conceptually linking project outcomes with conservation impacts remains difficult in complex systems. For example, sea turtles are one taxon in which unsustainable harvesting has been a particular problem. There have been a number of campaigns to reduce demand for sea turtle products, but we still have little evidence documenting their outcomes. We conducted the first formal impact evaluation of a conservation marketing campaign aimed at reducing the consumption of sea turtle meat and eggs. The campaign took place on the island of Sao Tome, Central Africa, and included traditional mass media advertisements as well as community events. This is one of few demand reduction evaluations that has assessed both human behaviour and biological conservation outcomes. It benefited from an advantageous setting for evaluation, as nesting sea turtles are relatively easy to monitor and the island's small size and geographic isolation increased the detectability of potentially confounding factors. We found a decrease in self‐reported sea turtle egg consumption and a decrease in poaching of adult sea turtles. However, multiple unforeseen difficulties arose which complicated attempts at causal attribution. We were hampered by spatial spillovers, design effects from the sensitive questioning technique, concurring law enforcement and changes in biological monitoring effort. These challenges highlight the difficulties faced by practitioners seeking to apply impact evaluations in the field. We reflect on what this means for future impact evaluations of behaviour change interventions in conservation. Our recommendations include combining multiple outcome measures to triangulate hard‐to‐measure behaviours and theory‐based evaluation methods to explore causal impacts.