Responsible Data and MERL

Monitoring, evaluation, research and learning (MERL) are by nature data-heavy activities. It makes sense, then, that over the past decade, the use of digital technology and digital data have permeated the practice of MERL.

In the social change sphere (humanitarian aid, development work, human rights, and program areas such as health, education, social protection, and protection overall), much of the data that we collect when conducting MERL comes from at-risk populations or underrepresented populations. This data influences decisions to support these same populations to access rights and services. It also tells us whether or not our interventions have worked and for whom. For this reason, a responsible data approach that takes data ethics and data protection into account is an imperative for MERL. 

Around 2013 or so, the use of digital tools and platforms to support MERL began to claim more attention among a small set of ‘early adopter’ MERL practitioners. We began seeing mobile devices used to collect data and as a way to encourage community feedback on programs. The use of satellite data and participatory mapping projects became more prominent, and there was great hope placed on crowdsourcing, and citizen journalism for gathering insights. In 2014, Michael Bamberger and I took a closer look, laying out an initial landscape of digital approaches to monitoring and evaluation. 

At the same time, following a discussion on the ethics of participatory mapping, a group of practitioners (including The Engine Room’s founders) assembled to look more closely at ethics in technology for development. We were concerned that development agencies were pushing innovation and technology in development while being largely unaware of data ethics, privacy and security issues that could expose individuals and communities to risk. Our first MERL Tech Conference also happened that year.

When adoption of innovation is used as the principal indicator of success or failure, the wider positive or negative ramifications, including unintended consequences, costs, and risks are likely to be overlooked, as noted by Glover. Close overlap with the Responsible Data community has helped ensure that the MERL Tech community continuously reflects on these concerns. We have created a space for dialog and discussion between tech developers, early adopters of technology in MERL, privacy and ethics advocates, and those who are newly learning about how technology can enable MERL. This has allowed the sector to improve its use of digital data, to highlight potential negative outcomes for vulnerable groups, and to introduce approaches to mitigate the harm that can come from the collection and use of MERL data.

The Responsible Data and MERL Tech communities have together explored many such areas of concern, including consent in the digital age, developing responsible data policies, and operationalizing responsible data policies. As the MERL community matured, discussions about RD moved from the margins to the center.

Over the years, we’ve seen heated debates about the potential for harm and unintended consequences stemming from digital approaches or from partnerships whose value and ethics are questioned. Together, the MERL Tech and RD communities advocated for ethical frameworks and better research on the potential for technologies and digital approaches to do harm. For example, research by Oxfam and The Engine Room on balancing the risks and benefits of using biometrics in the humanitarian field has helped to shape how we think about the role of emerging technology. We also co-curate a Responsible Data Resource List which lives on the MERL Tech site and on the RD website. Key to these discussions is the active and lively community debate that happens on the RD listserv,  at events by both organizations, and additional discussions that have taken place on these themes at the New York City Technology Salons. This constant reflection on the responsibilities of MERL professionals to use data in ethical and responsible ways has strengthened the MERL Tech space. 

While collective progress is being made in documenting and assessing technology-enabled MERL initiatives and good practice guidelines are emerging, ethical questions related to these new and emerging methods and approaches remain. In 2022, we are talking about much more than mobile data gathering, mapping and crowdsourcing, as we have explored in a series of MERL Tech State of the Field reports. Alongside more traditional uses of digital tools and data, we see ‘big data’ and predictive analytics sitting squarely in the MERL space. Emerging tools and technologies such as blockchain, artificial intelligence and machine learning, new forms of data storage, text and voice analytics, biometrics, non-traditional data and metadata are being explored as part of the MERL toolbox. Collaboration with the RD community helps the MERL Tech community ensure that we are looking at the ethics and RD issues that come with new and emerging approaches to MERL. 

Going forward, the MERL Tech community is addressing the fallout of COVID-19, which has made digital MERL and remote monitoring even more relevant. Building on a Responsible Data in MERL during COVID-19 series of events co-hosted with the CLEAR Center in Anglophone Africa), we have convened a Responsible Data in M&E – RDiME working group and community of practice that focus on these issues in the African context. Members of the working group developed guidance on data governance and responsible data practices for MERL with a focus on African contexts. We will also continue our focus on documenting, sharing, learning, training, and guidance on how to improve rigor, validity, representativeness, and inclusion, how to enhance the safeguarding of vulnerable individuals and groups and ways to assess new approaches and methods to ensure that ethics and safeguarding are included in MERL design and implementation. 

Ultimately, we hope our two communities will continue to collaborate in order to strengthen the MERL sector through intentional, responsible, and ethical approaches to technology and digital data.