Evaluation of Zambia Accountability Programme
Zambia remains one of the least-developed countries in the world. Despite recent economic growth, it continues to receive significant programme aid from DFID, and other organisations, to support human and social development, improved governance and wealth creation.
We were commissioned, in 2015, to undertake a four-year independent evaluation of the DFID funded, Zambia Accountability Programme (ZAP) which aims to encourage effective collaboration on government reform and development issues.
Operating at national, sub-national and local levels, the five-year ZAP focuses on improving service delivery, particularly in the health and education sectors; strengthening democratic process (with a specific focus on elections); and promoting inclusive growth and economic advocacy. ZAP is also working to strengthen broadcast and social media in Zambia to make them better equipped to hold the government to account on issues affecting society.
The purpose of our evaluation is to identify gaps in the evidence base in relation to democracy and accountability interventions and directly contribute to filling these gaps. We are achieving this by gathering evidence to test the validity of the Theory of Change. From this we are generating lessons and recommendations around what is and isn’t working, why, and in what contexts, and also assessing the programme’s value for money.
In view of the complex and adaptive environment of the ZAP, the evaluation design is framed within the Developmental Evaluation (DE) approach, with theory-based evaluation and elements of case-based evaluation at its core. This approach requires the evaluation team to work with the programme rather than as an external partner; thereby facilitating real-time feedback and continuous development during the programme. This requirement is carefully balanced with the need for the evaluation team to remain independent.
To support this, we have worked closely with the programme and DFID to develop an evaluation learning framework which maps key challenges faced by the programme and identifies what it and DFID want to learn from the process. The evaluation will take a cyclical approach with each cycle revisiting the evaluation learning framework to determine new and emerging lines of enquiry, before data collection begins.
Given the multi-dimensional nature of the programme’s planned outcomes, a combination of data collection methods are being employed. These include qualitative, and some quantitative, primary data collection, as well as review and analysis of secondary data sources. Following each cycle of data collection, we provide a synthesis of the analysis to allow the programme and DFID to make sense of the findings, and to discuss, where necessary, programme adaptation.