When experts disagree: the need to rethink indicator selection for assessing sustainability of agriculture

Sustainabilityindicators are well recognized for their potential to assess and monitor sustainabledevelopment of agricultural systems. A large number of indicators are proposed in various sustainabilityassessment frameworks, which raises concerns regarding the validity of approaches, usefulness and trust in such frameworks. Selecting indicators requires transparent and well-defined procedures to ensure the relevance and validity of sustainabilityassessments. The objective of this study, therefore, was to determine whether experts agree on which criteria are most important in the selection of indicators and indicator sets for robust sustainabilityassessments. Two groups of experts (Temperate Agriculture Research Network and New Zealand Sustainability Dashboard) were asked to rank the relative importance of eleven criteria for selecting individual indicators and of nine criteria for balancing a collective set of indicators. Both ranking surveys reveal a startling lack of consensus amongst experts about how best to measure agricultural sustainabilityand call for a radical rethink about how complementary approaches to sustainabilityassessments are used alongside each other to ensure a plurality of views and maximum collaboration and trust amongst stakeholders. To improve the transparency, relevance and robustness of sustainableassessments, the context of the sustainabilityassessment, including prioritizations of selection criteria for indicator selection, must be accounted for. A collaborative design process will enhance the acceptance of diverse values and prioritizations embedded in sustainabilityassessments. The process by which indicators and sustainabilityframeworks are established may be a much more important determinant of their success than the final shape of the assessment tools. Such an emphasis on process would make assessments more transparent, transformative and enduring.