M&E Systems and Data Tools
Firdale has designed and built M&E data systems and tools for foundations, non-profits and impact investors. These have included data collection, storage, data cleaning code, automated reports and live data dashboards.
Integrated M&E System: surveys, data storage and dashboard for a tertiary institute
Firdale Consulting hosted interactive workshops to update TSiBA Business School's Theory of Change, designed surveys to collect data for key indicators of the TOC, uploaded surveys to Google Workspace, and designed and programmed an online data dashboard linked to the back-end data system of the institute. The dashboard enabled TSiBA's staff to analyse education and financial data in one place in real-time. Firdale developed a scoring system for each indicator to simplify the complex data into easy-to-interact with visuals. The dashboard included TOC heat-maps that Firdale designed to visually depict progress. Training was provided on how to use the surveys and dashboard and how to analyse data.
Dashboard to evaluate student and staff outcomes for an education NGO
Firdale worked with a national education NGO to develop a live data dashboard with multiple tabs and slicers to enable the NGO staff to track their progress against various indicators. These included academic results, throughput rates to further education, attendance rates, and ratios of educators to students during the programme, amongst other indicators. The dashboard user could slice the data across variables such as gender, location or date, and could view the data at a single time point or look at the trend over time. The NGO staff were provided with training on how to use the dashboard.
Tobacco control dashboards in various African countries
Megan Little, Firdale Consulting's Director, is working with the Research Unit on Excisable Products at the University of Cape Town, and USA-based data solutions company, Development Gateway, on a 3.5 year project for the Bill and Melinda Gates Foundation in the field of tobacco control. The project began in January 2020. In six to eight different African countries, we are conducting comprehensive in-country stakeholder meetings and workshops with the leading tobacco control stakeholders in government, civil society and academia; conducting a comprehensive data review; designing data collection models and collecting raw data; and designing and building an online tool that synthesises all tobacco control data into an easy-to-use online tool for stakeholders.
Integrated M&E System: surveys, data storage and dashboard for a rural NGO
Firdale worked with the award winning NGO, Bulungula Incubator, to update their M&E framework. This included updating their Theory of Change, creating a comprehensive indicator matrix, conducting a widespread data mapping and cleaning exercise and designing new surveys across their multiple programmes to elicit new information in line with their TOC. Thereafter we reviewed a variety of software for data collection, management and analysis and designed a bespoke solution for the client that was cost-effective and tailored to their needs, which included data collection through surveys, data storage and a live data dashboard that we designed and built using R. A detailed Case Study of this project is available here.
Automated data analysis and report generation for ECD report
In 2018/9 Firdale was hired by Innovation Edge and worked with a prominent group of ECD experts to design a simple report template for the various NGOs participating in the Early Learnings Outcomes Measure testing programme. The report included simple data analytics which effectively conveyed the key findings to the target audience. In addition, using the data analytics software Stata, we wrote code to allow the team to automatically generate the report content when new data were submitted. See the Case Studies page for a detailed look at this project.
Online survey for a performing arts NGO
We worked with Magnet Theatre, a performing arts NGO, to update their current data capturing system through a comprehensive online survey. This required working closely with the NGO staff to understand the outputs and outcomes they needed measured, designing the survey questions and sampling frame, and uploading the survey onto Google Forms. At the end of this process, the NGO staff were trained on how to make changes to the online survey and analyse the results by exporting the results into Excel or using the simple analytics interface embedded into Google Forms.
Evaluations, Reports and Research
Firdale Consulting has conducted quantitative and qualitative evaluations, research and reports across a number of sectors, including education, early childhood development, technology, employment in the green economy and youth development.
The Firdale Consulting team was appointed as the M&E consultants for the Green Outcomes Fund, a three year Jobs Fund project that began in January 2020 and focuses on the green SMME sector. The Jobs Fund project partners are GreenCape and Bertha Centre. In the project, four Catalytic Finance Partners are investing in green SMMEs across South Africa with the goal of stimulating growth and employment in the sector.
We are tracking performance using a combination of qualitative and quantitative data (including surveys, interviews and focus groups). We have designed and built an interactive data dashboard and will deliver three comprehensive evaluation reports over the course of the project. This project will be discussed in detail in one of our forthcoming Case Studies.
Quantitative evaluation of academic data
In partnership with JET Education services, Firdale evaluated a national school governance programme to understand the impact of the programme on students' academic marks and throughput rates. The retrospective impact evaluation required quantitative data analysis through developing a propensity score match on a variety of independent variables, and thereafter running a difference-in-difference model. This model compared the change in academic results over the course of the programme for the programme schools versus the matched schools. These quantitative results were coupled with in-depth qualitative analysis conducted by the JET consultants.
Quantitative and qualitative evaluation of a mobile app for ECD practitioners
Firdale conducted a quantitative and qualitative evaluation of two mobile apps and the technology platform that created them for Innovation Edge, a prominent South African donor. The process included a Theory of Change workshop with each of the partners, the development of comprehensive indicator matrices linked to the TOC, a desktop literature review to benchmark statistics on similar products, the coordination of multiple focus groups across South Africa to obtain rich qualitative data, deep analysis of the back-end app data, a user experience and technology review by external experts, and finally the compilation of a comprehensive 100 page evaluation report with recommendations for each partner to improve their offering.
Evaluation and M&E system review for a national literacy programme
Firdale conducted a comprehensive evaluation of Nal'ibali, a national NGO focusing on literacy. Nal'ibali had been collecting data which were stored in disparate sources and the evaluation required a full data review and the cleaning and merging of the datasets into one consolidated dataset. Thereafter the data were analysed to measure a number of performance indicators to understand progress to date against their funder's targets. The analysis highlighted a number of limitations regarding the data collection process and recommendations were provided for improving data collection in the short/medium term. In addition, we reviewed the current M&E system and provided recommendations on how this could be improved.
Evaluation and M&E system review for youth development programme for at-risk youth
We were appointed to explore the impact an educational programme on outcomes for at-risk youth. We reviewed the existing data, cleaned and merged the data into a consolidated dataset and analysed the data to provide a detailed evaluation. The evaluation reported on the characteristics of the participants with regards to socio-economic status, alcohol and drug abuse, prior education, age, gender and geographic location. The outcomes included educational outcomes during the programme as well as longer term job outcomes. Finally, a simple, cost effective data capturing system was designed for the client.