The 5 steps to survey design
Previous Projects
M&E Systems and Data Tools
Firdale has designed and built M&E data systems and tools for foundations, non-profits and impact investors. These have included data collection, storage, data cleaning code, automated reports and live data dashboards.
Monitoring, Evaluation and Implementation Framework for the South African National Aids Council
In 2021 Firdale Consulting worked with Health Focus South Africa to develop a Monitoring, Evaluation and Implementation Framework for the South African National Aids Council (SANAC) for Adolescent Boys and Young Men. This included developing a Theory of Change, Indicator Matrix, a data review, and a reporting framework. We focused on biomedical, structural and behavioural factors.
The Health Focus team reflected that Firdale’s “extensive research into extracting data from a wide variety of sources to establish the evidence base for the M&E framework, in addition to the interactive frameworks and reporting tools” exceeded their expectations. In addition, they indicated that “Firdale demonstrated a high degree of professionalism, skill and experience [and] worked beyond the project scope timeframes to ensure delivery against the set objectives.”
Design and Implementation of an M&E Eystem for an Education Non-Profit
VVOB – education for development is an international non-profit that contributes to an equitable world in which governments are capacitated to fulfil the rights to education for every learner. They operate in South Africa, Kenya, Rwanda, Uganda, Zambia, Cambodia, Vietnam and Ecuador and have their head offices in Belgium. Firdale Consulting has been working with VVOB since early 2021 to strengthen, build and maintain their M&E system. This has included reviewing their M&E system (including the theory of change, existing data and collection tools); compiling an indicator matrix; cleaning the historical data; reviewing and selecting software for their new M&E system; integrating their existing system with the new software; providing guidance for data entry; designing and building a dashboard; and providing training on data analysis and software such as Stata and PowerBI. In our work we conduct various workshops both online and in-person
Integrated M&E System: surveys, data storage and dashboard for a tertiary institute
Firdale Consulting hosted interactive workshops to update TSiBA Business School's Theory of Change, designed surveys to collect data for key indicators of the TOC, uploaded surveys to Google Workspace, and designed and programmed an online data dashboard linked to the back-end data system of the institute. The dashboard enabled TSiBA's staff to analyse education and financial data in one place in real-time. Firdale developed a scoring system for each indicator to simplify the complex data into easy-to-interact with visuals. The dashboard included TOC heat-maps that Firdale designed to visually depict progress. Training was provided on how to use the surveys and dashboard and how to analyse data.
Dashboard to evaluate student and staff outcomes for an education NGO
Firdale worked with a national education NGO to develop a live data dashboard with multiple tabs and slicers to enable the NGO staff to track their progress against various indicators. These included academic results, throughput rates to further education, attendance rates, and ratios of educators to students during the programme, amongst other indicators. The dashboard user could slice the data across variables such as gender, location or date, and could view the data at a single time point or look at the trend over time. The NGO staff were provided with training on how to use the dashboard.
Tobacco control websites in six African countries for the Gates Foundation
The Firdale Consulting team is working on the Tobacco Control Data Initiative (TCDI) project which is a 4 year project that began in January 2020 in six African countries (The Democratic Republic of Congo, Ethiopia, Kenya, Nigeria, South Africa and Zambia). The project is led by Development Gateway in partnership with The Research Unit on Excisable Products at the University of Cape Town and is funded by the Bill and Melinda Gates Foundation.
We are conducting comprehensive in-country stakeholder meetings and workshops with the leading tobacco control stakeholders in government, civil society and academia; conducting a comprehensive data review; designing data collection models and collecting primary data; and designing and building online websites that synthesises all tobacco control data into an easy-to-use online tool for stakeholders. Check out the South African TCDI website here and the Nigeria TCDI website here.
Integrated M&E System: surveys, data storage and dashboard for a rural NGO
Firdale worked with the award winning NGO, Bulungula Incubator, to update their M&E framework. This included updating their Theory of Change, creating a comprehensive indicator matrix, conducting a widespread data mapping and cleaning exercise and designing new surveys across their multiple programmes to elicit new information in line with their TOC. Thereafter we reviewed a variety of software for data collection, management and analysis and designed a bespoke solution for the client that was cost-effective and tailored to their needs, which included data collection through surveys, data storage and a live data dashboard that we designed and built using R. A detailed Case Study of this project is available here.
Automated data analysis and report generation for ECD report
In 2018/9 Firdale was hired by Innovation Edge and worked with a prominent group of ECD experts to design a simple report template for the various NGOs participating in the Early Learnings Outcomes Measure testing programme. The report included simple data analytics which effectively conveyed the key findings to the target audience. In addition, using the data analytics software Stata, we wrote code to allow the team to automatically generate the report content when new data were submitted. See the Case Studies page for a detailed look at this project.
Online survey for a performing arts NGO
We worked with Magnet Theatre, a performing arts NGO, to update their current data capturing system through a comprehensive online survey. This required working closely with the NGO staff to understand the outputs and outcomes they needed measured, designing the survey questions and sampling frame, and uploading the survey onto Google Forms. At the end of this process, the NGO staff were trained on how to make changes to the online survey and analyse the results by exporting the results into Excel or using the simple analytics interface embedded into Google Forms.
Evaluations, Reports and Research
Firdale Consulting has conducted quantitative and qualitative evaluations, research and reports across a number of sectors, including education, early childhood development, technology, employment in the green economy and youth development.
Evaluation of a multi-year, national Jobs Fund project
The Firdale Consulting team was appointed as the M&E consultants for the Green Outcomes Fund, a three year Jobs Fund project that began in January 2020 and focuses on the green SMME sector. The Jobs Fund project partners are GreenCape and Bertha Centre. In the project, four Catalytic Finance Partners are investing in green SMMEs across South Africa with the goal of stimulating growth and employment in the sector.
We are tracking performance using a combination of qualitative and quantitative data (including surveys, interviews and focus groups). We have designed and built an interactive data dashboard and will deliver three comprehensive evaluation reports over the course of the project. This project will be discussed in detail in one of our forthcoming Case Studies.
Randomised Control Trial to Motivate Antenatal Care Seeking Behaviour
In 2015, Laura Rossouw designed, implemented, monitored and evaluated the Thula Baba Box Randomized Controlled Trial for Stellenbosch University. The RCT measured the impact of various interventions in motivating earlier antenatal care seeking behaviour among South African women in a peri-urban environment. Rossouw developed all the interventions based on prevailing literature, her own qualitative research and drawing strongly on various behavioural economic theories. The research was conducted alongside Professors Ronelle and Rulof Burger, and has since been published in international journals:
Rossouw, L., Burger, R.P. and Burger, R., 2019. An incentive-based and community health worker package intervention to improve early utilization of antenatal care: Evidence from a pilot randomised controlled trial. Maternal and Child Health Journal, 23(5), pp.633-640. https://link.springer.com/article/10.1007/s10995-018-2677-9
Rossouw, L., Burger, R.P. and Burger, R., 2021. Testing an Incentive-Based and Community Health Worker Package Intervention to Improve Maternal Health and Nutrition Outcomes: A Pilot Randomized Controlled Trial. Maternal and Child Health Journal, 25(12), pp.1913-1922. https://link.springer.com/article/10.1007/s10995-021-03229-w
Economic Evaluation of the Umbiflow Doppler to Manage Pregnancies
In 2016, Laura Rossouw conducted a cost-effectiveness analysis of the Umbiflow Doppler to screen and manage a low-risk pregnant population in South Africa. The research was based on a study designed and implemented by Professor Bob Pattinson and Doctor Spencer Nkosi at the South African Medical Research Council.
Quantitative evaluation of academic data
In partnership with JET Education services, Firdale evaluated a national school governance programme to understand the impact of the programme on students' academic marks and throughput rates. The retrospective impact evaluation required quantitative data analysis through developing a propensity score match on a variety of independent variables, and thereafter running a difference-in-difference model. This model compared the change in academic results over the course of the programme for the programme schools versus the matched schools. These quantitative results were coupled with in-depth qualitative analysis conducted by the JET consultants.
Study on Addressing the Failure to Initiate TB Treatment
In 2017, Laura Rossouw was hired by Stellenbosch University to consult on a study that addresses the failure to initiate TB treatment after testing at Primary Health Care facilities in Cape Town, South Africa. The study was conducted jointly with researchers from various institutions, including Erasmus University (the Netherlands). Laura was largely consulting on instrument development, especially in monitoring the behavioural health economic determinants of health-seeking behaviour. The research has subsequently been published:
Burger, R., Caldwell, J., Claassens, M., Mama, K., Naidoo, P., Rieger, M., Rossouw, L., van Doorslaer, E. and Wagstaff, A., 2021. Who is more likely to return for TB test results? A survey at three high-burden primary healthcare facilities in Cape Town, South Africa. International Journal of Infectious Diseases, 113, pp.259-267.
Quantitative and qualitative evaluation of a mobile app for ECD practitioners
Firdale conducted a quantitative and qualitative evaluation of two mobile apps and the technology platform that created them for Innovation Edge, a prominent South African donor. The process included a Theory of Change workshop with each of the partners, the development of comprehensive indicator matrices linked to the TOC, a desktop literature review to benchmark statistics on similar products, the coordination of multiple focus groups across South Africa to obtain rich qualitative data, deep analysis of the back-end app data, a user experience and technology review by external experts, and finally the compilation of a comprehensive 100 page evaluation report with recommendations for each partner to improve their offering.
Evaluation and M&E system review for a national literacy programme
Firdale conducted a comprehensive evaluation of Nal'ibali, a national NGO focusing on literacy. Nal'ibali had been collecting data which were stored in disparate sources and the evaluation required a full data review and the cleaning and merging of the datasets into one consolidated dataset. Thereafter the data were analysed to measure a number of performance indicators to understand progress to date against their funder's targets. The analysis highlighted a number of limitations regarding the data collection process and recommendations were provided for improving data collection in the short/medium term. In addition, we reviewed the current M&E system and provided recommendations on how this could be improved.
Evaluation and M&E system review for youth development programme for at-risk youth
We were appointed to explore the impact an educational programme on outcomes for at-risk youth. We reviewed the existing data, cleaned and merged the data into a consolidated dataset and analysed the data to provide a detailed evaluation. The evaluation reported on the characteristics of the participants with regards to socio-economic status, alcohol and drug abuse, prior education, age, gender and geographic location. The outcomes included educational outcomes during the programme as well as longer term job outcomes. Finally, a simple, cost effective data capturing system was designed for the client.
