The 5 steps to survey design
The 5 steps to survey design
Tyryn Carnegie
We’ve looked at the importance data have for M&E in our previous blogs. Sometimes the data we need are already available (for example local statistics collected by government). At other times, data need to be collected and one way to do this is through a survey.
What are surveys? It’s almost certain that you have answered a survey before - for example a customer satisfaction survey at a restaurant or a government census. A survey (or questionnaire) is a set of questions which aims to extract specific information from the chosen respondents such as their attitude, preference or knowledge. This blog post introduces surveys, a broad subject with many components, each of which we’ll be discussing at more depth in later posts.
Below we depict the chain of events that occur when surveys are undertaken.
1. Start by choosing your survey type
Depending on what data are needed for an organisation’s required indicators, surveys can be broken up into two broad types: cross-sectional and longitudinal (or panel) surveys. In cross-sectional surveys respondents are interviewed at a single point in time whilst longitudinal surveys track the same respondents and ask them broadly the same set of survey questions each time. While longitudinal surveys tend to offer very rich data, tracking respondents requires a larger budget and more careful field management. As such, cross-sectional surveys are more common.
2. Next pick your respondents
Survey sampling describes the process of selecting a sample of people to answer the survey from the broader target population. As an example, to assess the impact of an intervention targeting 10 000 people, it may only be necessary to survey 500 of those people to accurately quantify the programme’s impact. Surveys are costly to conduct (in both time and resources), so the purpose of sampling is to reduce the costs required to survey the entire population. The sampling process needs to be carefully planned, as if the sample is not an accurate representative of the target population, the usefulness of the findings is diminished. It is usually worthwhile to ask a statistician for assistance in the sampling.
3. Get into the detail and write your questions
When designing the survey questions, one needs to tailor them to capture the outcomes of interest and ensure that all relevant indicators from your indicator matrix are reflected. Important considerations include unique identifiers (per respondent), question order, survey length, user-friendliness, phrasing, and the use of techniques to minimize mistakes on the part of the respondent. We will be discussing question design in more depth in future blogs. The choice of respondent is also crucial: the questions may be good, but the person who is responding may not have the necessary knowledge to answer accurately.
4. Choose and train your fieldworkers carefully
After the questions have been set, they of course need to be answered. This can be done through distance surveying, with interviews done over the phone or respondents filling in an online form, or by in-person interviews with a surveyor asking respondents questions and recording their answers face-to-face.
The quality of the staff has a significant impact on the collected data and proper training needs to occur and careful field management should be conducted to ensure quality data. In order to maximize response rates, visits should be scheduled carefully and non-respondents should be followed up on wherever possible.
5. Finally, capture the survey responses
Lastly, how the data are captured is an important consideration and something that has changed as technology has improved. Survey responses were initially captured by pen and paper forms (a practice still widely used at various NGOs today). There has however been an increase in the use of electronic data capturing forms such as mobile apps which increase the accuracy and speed at which answers are captured.
In the next post we explore the differences between cross-sectional and longitudinal surveys and learn about the importance of unique identifiers.