The 5 steps to survey design
The 4 elements of an effective M&E system
Megan Little
Running a non-profit is extremely demanding, with each day posing new challenges for non-profit leaders and their staff. The workforce needs to become adept at switching between multiple tasks, which can include writing new funding proposals, managing relationships with existing donors, serving participants, managing finances, recruiting new staff, and much more.
However, the one item that is almost universally avoided is data analysis and reporting. Most non-profit staff do not come from a quantitative or software background and data analysis feels both intimidating and boring.
Over time, without a good monitoring & evaluation (M&E) system, non-profit data tend to multiply in complexity, creating a mess of hardcopy papers and spreadsheets. When forced to analyse data for a donor report, staff may struggle to find the data they need, and many fear ‘getting it wrong’ and generating the wrong statistics.
A poor M&E system for non-profits often leads to:
-
Stressed out and demotivated staff
-
Lack of clarity on programme impact and direction
-
Wasted time putting together reports
-
Frustration from donors who want to see results
Look to the ABCS for an effective non-profit M&E system
To address these challenges, non-profits need an M&E system that provides information in real time. A good M&E system needs to cover each of the ‘ABCS’.
A non-profit M&E system should:
-
Be accessible to all key stakeholders
-
Have buy-in from all relevant parties
-
Be cost-effective and
-
Be specific to non-profits' needs
1. Accessible
Many non-profit staff feel intimidated by all things data. Knowing which data to collect, how frequently to collect, and how to sample can feel daunting. Understanding how to navigate the data and obtain the results and graphs needed often feels nearly impossible. As such, many non-profit staff fail to engage with their own data.
The biggest mistake I have seen non-profits make is to either place full responsibility for data analysis on one person in the organisation, or outsource data analysis entirely to an external consultant. In both scenarios, this alienates everyone else in the organisation and the organisation is left stumbling in the dark, without their guiding light, data.
A good M&E system must be accessible to all key stakeholders. How can this be achieved? Speak to the users to understand their constraints, and then build the system with these in mind.
Constraint: "I don't understand statistics or data analysis"
Solution: A simple dashboard with easy to understand graphics
Constraint: “I don’t have Wi-Fi or cellphone data to access an online dashboard”
Solution: Automated reports and infographics shared via email or WhatsApp
2. Buy-in
Without staff buy-in, entire M&E systems can fail. Non-profit staff need to see how the M&E system will make their lives easier, and propel them towards achieving the organisation’s overall objective, otherwise the system will languish unused.
This is for a variety of reasons:
-
M&E systems rely on consistent data capturing, which can be tedious and time consuming. If staff do not see the value in their work, corners will be cut.
-
Non-profit employees often prefer to focus on service/product delivery to those in need, rather than data collection/management.
-
An M&E system is often seen as a donor requirement, not something that could be useful for managing day-to-day operations.
The solution is to engage with all key stakeholders when designing the M&E system, understand their individual needs, create an M&E system that will be useful to them (#4 Specific), and ensure it is easy to use (#1 Accessible).
3. Cost effective
An effective M&E system should allows its users to:
-
Capture data
-
Manage and clean data
-
Analyse data
However, such systems can be costly to build, customise, license and deploy.
There are a wide variety of M&E systems that have been created to support non-profits and their funders. Some of these platforms offer all three services (data collection, management and analytics) whilst others provide niche offerings.
Niche products have an added challenge because the 3 core functions of a good M&E system need to connect seamlessly, such that all collected data can be cleaned and analysed easily. This is often not possible with disparate products.
Many of these products are expensive to license, and have hidden costs as one needs to hire consultants to set them up, add/adjust elements over time, provide training on how to use the product, and troubleshoot over time. Furthermore, many products bundle in a range of functions, many of which are not useful to non-profits. This increases the price without actually increasing usability.
Many of the organisations we have worked with find one-size-fits all M&E systems do not match their needs and as such stop using them.
We have reviewed various M&E system providers, but are yet to find an affordable, easy to use platform that addresses the widely varied needs of non-profit users.
The solution is an M&E system designed for the non-profit’s specific needs that uses open-source software, publicly available APIs and has minimal licensing fees. This ensures the product is useful and keeps costs low.
4. Specific
In order to get stakeholder support for an M&E system, it is important that the system is useful and specific to user needs. This requires speaking to all key stakeholders about their challenges at work and brainstorming how data will alleviate these challenges. It also requires understanding the stakeholders’ data literacy, internet availability, reporting cycles, and overall non-profit model. Getting these issues right will ensure that the design of the data system is as user-friendly as possible. Let’s look at some examples.
Organisation A coordinates health-worker home-visits in poor communities. A useful M&E system could include:
-
Customisable surveys for health-workers to complete with households.
-
Reminders to follow up with specific households.
-
Summary statistics presented in simple graphics to export for donor reports.
-
Surveys
Organisation B created a mobile app with lesson plans for ECD practitioners. A useful M&E system could include:
-
A dashboard to track app usage by recipient type, location, language.
-
Automatic reminders to follow-up with users that have stopped using the app.
-
Automated reports showing changes in usage over time, to send to donors.
-
Automated Reports
If Organisation A and B had adopted a one-size-fits-all M&E system, it is highly unlikely the end-product would have addressed their individual needs. Non-profits offer highly varied products and services to diverse beneficiaries, and effective M&E systems need to take this into account.
Conclusion
Choosing the right M&E system for your non-profit can be challenging. There are a wide variety of M&E products available, and it is useful to keep in mind the ABCS when picking the right system. Ensure that the system will be Accessible to all potential users, secure buy-in from key stakeholders, find a solution that is cost-effective and make sure the system design is specific to the needs of the organisation.
An effective M&E system should empower its users with up-to-date information showing how well the programme is being implemented and the impact it has had. It should reduce the burden of data collection, analysis and reporting for the team, thus liberating them to focus on what they do best. In the words of Kofi Annan, “Knowledge is power. Information is liberating”.