This blog was written by Paul Stanchfield, Fintrac’s Director of Monitoring and Evaluation. Paul oversees all M&E activities for Fintrac’s USAID flagship projects, implements standardized M&E methodologies and associated training/capacity building across the project portfolio, and advises the executive team, senior field staff and managers on strategic corporate M&E issues.

Responding to the COVID-19 pandemic has challenged us to be more flexible, adaptable, and creative. For Fintrac M&E teams, this includes responding to canceled in-person surveys, which are critical tools in assessing our progress and impact, by adapting the process and tools to work in a virtual scenario. Hopefully the lessons we’ve learned so far can be helpful to practitioners around the world when navigating similar transitions or challenges.

1. Develop a Plan A, B… and even C: As COVID-19 impacts staff and contractor mobility at all levels, the development of a contingency plan is essential when planning data collection. This is even more critical if field work is planned three or four months out, because countless variables can (and will likely) change. Therefore, when developing your response plan, consider multiple scenarios – from highly conservative approaches that require all virtual interviews, to in-person approaches that integrate social distancing. Prioritize each plan to account for the potential that any data collection component might quickly shift to phone-based surveys, including key informant interviews (KIIs).

Closely monitor regional and municipal-level COVID-19 data released by credible authorities, and then cross-check it with your project location. Doing so can help you understand if there are any elevated risks. Likewise, closely monitor your updated budgets and work plans to better understand the cost and schedule implications of your strategies, especially if you need to adjust travel plans due to elevated risk. For instance, when shifting to phone-based interviews you will save on enumerator transportation costs, per diems, and lodging. At the same time, however, you will likely offset some of those savings through much higher phone bills and other unforeseen costs associated with hiring community representatives to help contact and organize respondents. Regardless of the selected scenario, adding a line item for personal protective equipment is a must.

2. Reassess your M&E subcontractor: If you are working with an M&E subcontractor, re-evaluate their ability to collect data in person, on the phone, or through a hybrid approach. You may find that some subcontractors are uniquely organized to shift to a more centralized, virtual data collection scenario while others may have a well-developed network of regionally-based enumerators and supervisors. Some subcontractors may even have call centers in place with robust protocols for data security and remote monitoring and supervision. In some cases, you may also want to review your contracting mechanism to assess if there are any benefits from shifting from fixed priced to cost reimbursable contracts.

3. Trim your survey instruments: Depending on travel restrictions, field teams will likely need to shift to collecting some, if not all, data via phone surveys. This scenario presents its own series of unique challenges and limitations. Without a doubt, it’s more difficult to develop a rapport, verify practices and key documents, and observe and respond to non-verbal cues on the phone. Data collectors can expect countless missed calls, wrong and outdated numbers, weak signals, and dropped calls when a respondent’s phone runs out of batteries mid-survey. Enumerators have the added task of maintaining the respondent’s engagement while estimating yields, calculating sales, or gathering a laundry list of input cost data.

Without adjustments, the original 90-minute in-person survey can easily become a three-hour phone survey labyrinth. This is simply not feasible. To minimize the burden on enumerators and respondents alike, work early to cut out non-essential questions from your survey instruments. Review each question one-by-one, using a red marker to highlight questions that do not directly contribute to either a performance indicator or offer significant insights for use by the project implementation team. Now is the time to cull those “nice-to-have” perception questions, unwieldly household rosters, and long-winded exploratory questions that have been collecting dust for years. Consider recrafting longer questions and shifting to more closed and scaled responses to make the interview more phone friendly. With that said, you may want to incorporate a few timely must-have questions into your survey that are focused on understanding the local context and impact of COVID-19 on the respondent’s daily life.

3. Verify beneficiary phone numbers early: All M&E and program staff are acutely aware of the challenges that come with maintaining accurate and complete lists of beneficiary phone numbers. However, we usually don’t depend on the lists when we do in-person surveys. Therefore, start reviewing and cleaning these lists sooner rather than later. Work with your database manager, regional M&E specialists, and implementation teams as early as possible to review the completeness and accuracy of beneficiary phone numbers.

As a best practice, complete checks on the full list of beneficiaries and on those individuals eventually selected as part of your sample. Aside from the obvious blank or incorrect phone numbers, you’ll also likely come across phone numbers for the beneficiary’s cousin in the next town over. Or, you might discover that the beneficiary has changed their phone number to a different carrier altogether. Entire communities may also have limited or non-existent cellular signals. In addition, verifying phone numbers early on can help better understand sampling-related adjustments required for non-response, and any additional steps required for reaching individuals without cellular service (like hiring coordinators in the communities to facilitate the time and place for calls).

Adjust enumerator training, focusing on digestibility: Shifting to phone-based data collection requires not only adjustments to data collection techniques but also reformulating both the duration and content of enumerator training. Given that these trainings are now conducted through virtual sessions, full-day enumerator trainings are too long and unwieldy. Instead, replace them with more digestible two- or three-hour training blocks that incorporate video calls and frequent checks for learning. Enumerators will also require training on new skills ranging from how to correctly follow phone-based informed consent procedures and introductory scripts to strategies to effectively plan and optimize time spent on the phone with producers who may only be available early in the morning or late at night.