Open Letter to WFP re: Palantir Agreement


/ February 8, 2019

TO: David Beasley, Executive Director, WFP

CC: Enrica Porcari, Chief Information Officer, WFP; Pierre Guillaume Wielezynski, Chief of Digital Transformation Services, WFP; Patricia Ann Kikuko Harvey, Inspector General and Director of Oversight Office, WFP

________________________________________________

Dear Mr. Beasley,

We are deeply concerned by the recent announcement from the United Nations World Food Programme (WFP) about a new data partnership with the private software company Palantir Technologies Inc. (Palantir). This partnership has the potential to seriously damage the reputation of the WFP, and its ability to achieve its mandate. Moreover, the partnership has the potential to seriously undermine the rights of 90 million people the WFP serves.

We write to urge you to reconsider the terms and scope of the agreement with Palantir, to be more transparent about the process and terms of the agreement, and to take concrete, immediate steps to mitigate the serious risks of harm arising from the agreement, including by establishing an external committee of experts to review the implementation of the partnership terms.

We, too, are keen for a debate centred around facts not speculation – but due to a significant lack of transparency, it is difficult to understand Palantir’s business practices and the risks they entail, as well as what WFP has partnered with them to do. From the little that we know, we can assume that the risks in working with them arise not from data sharing, but from the models that are extracted from WFP’s data, the software that is used, and biases within models that will be applied to the data.

Some of the more salient risks of concern to us include:

De-anonymization. There are a number of compounding risks that arise when huge datasets are merged and analyzed, which is the premise behind Palantir’s business model. Even when data points are stored separately, the “mosaic effect” presents a particular risk. Harvard researchers have noted that traditional privacy and anonymization frameworks that focus on identifying and removing personally identifiable information (PII) are “unsustainable and ineffective.” Even without full data being shared, significant risks arise from sharing metadata. WFP’s insistence that no personally identifiable information will be shared is largely invalid because of this point.

Bias. Another risk involves the use of algorithms to analyze data and flag “potential misuse,” a stated goal of the partnership. Based on past business practices, we assume Palantir will be using data models generated from their work with other organisations, and it is impossible to know what biases may be baked into these models. Algorithmic filtering is known to have a high risk of embedded biases and produce results of extremely variable quality and reliability. This is especially true in the context of data concerning widely diverse populations.

Rights to data, models and derivative analyses. The latest statement from WFP states that no access to WFP data will be provided under this agreement, and that WFP retains full control over the data, analysis and derivative work. However, it fails to explain what “control” implies in a legal sense, or whether it covers models, training data, and/or inferences made by such models. Intellectual property rights of any data, models or derivative analyses must be retained by WFP, or the WFP risks significant financial and intellectual property losses.

Future costs to WFP and sustainability. While the agreement may save money in the short term, long term costs should also be evaluated. Costs of using the system may increase over time, as experienced by other Palantir customers, who have complained about Palantir’s opaque pricing model, spiraling prices and a failure to deliver products. In addition, any future decision to end the partnership could be extremely costly and difficult, as the datasets will be difficult to extract and decouple from Palantir’s software.

Undermining the humanitarian principles. The partnership risks undermining WFP’s fundamental humanitarian principles and other core sector standards. For example, the Principles for Digital Development are a powerful force for the transparent, inclusive and equitable use of technology within the aid and development community. The WFP is a prominent and influential signatory to the Digital Development Principles, which means they have committed to “embody the concepts of the Digital Principles, represented in our work culture and in the policies and processes guiding our international development activities.”

Transparency and accountability. Nothing has been transparently shared about the procurement process that WFP set up to engage Palantir. Given the gravity of these concerns, building in transparent checks and balances – such as third-party audits, open procurement, and contract or agreement transparency, seems essential to ensuring that WFP can build upon the knowledge and expertise of experts in this community who want to support innovation and reduce harm. The “checks and balances” that are mentioned by WFP in their latest statement are done without any source documents being shared, making them closed-door agreements rather than actual checks and balances that they can be held accountable to. This sets a dangerous precedent for the WFP and other humanitarian agencies.

The undersigned organizations and individuals are dedicated to upholding human rights protection around the world. Many of us have worked firsthand with vulnerable populations, including with respect to gathering, sharing, and analyzing data about them in efforts to ensure basic rights and provide much-needed humanitarian aid. We recognize both the value that data can bring to making humanitarian programs more effective as well as the potential risks to those whose data is being gathered.

We recognise the complexity of the environment in which humanitarian agencies work, and the pressures and constraints that you face. In particular, we understand the constant pressure from donors to increase efficiencies and the effective use of every dollar. We are committed to supporting the responsible use of data in the sector, from developing strong, trusted partnerships, to developing better practices for information management.

If, as stated in the latest account, WFP is committed to engaging in responsible data practices, we urge you to do so in a transparent and accountable way, engaging in a meaningful way with the community here.

In conclusion, we urge you to take the following actions:

  1. Release the terms of the agreement in a show of transparency, and commit to sharing contracts with private sector companies in the future;
  2. Release information about the procurement and due diligence processes that went into deciding to engage with Palantir, as well as an assessment of Palantir’s compliance with the UN Business and Human Rights Guidelines;
  3. Establish an independent review panel to review the project plan and safeguards; and
  4. Take any necessary steps to amend the agreement to ensure the privacy and security of the people WFP serves, such as by:
    • Limiting the ability of Palantir to apply models and analyses from WFP’s data to other datasets;
    • Ensuring that WFP is not “locked-in” to the Palantir system by retaining control and rights over metadata and models built on the WFP data;
    • Demonstrating openness and establishing a consultation process open to the community;
    • Establishing a transparent grievance mechanism for those who may wish to challenge characterizations made based on data analysis;
    • Undertaking human rights and conflict sensitivity impact assessments; and
    • Establishing a clear protocol and agreement with Palantir for terminating the agreement should it become clear that the privacy and security of the people WFP serves cannot be ensured.

These recommended actions also pertain to other humanitarian and public interest organizations that have signed agreements with Palantir, including the Mercy Corps, NCMEC, C4ADS, Carter Center, Team Rubicon, the Enough Project, and the Rockefeller Foundation.

Sincerely,

AI Now Institute

Sarah Aoun

ARTICLE 19

Aspiration

Marc Beebe

Build Up

Kevin Clark

Code Innovation

DATACTIVE

Data Justice Lab

DataKind

DataKind UK

DensityDesign Lab

Derechos Digitales · América Latina

Daniel Dietrich

Digital Civil Society Lab

William Doran

Doteveryone

The Engine Room

Equality Labs

Global Justice Clinic, NYU School of Law

Lauri Goldkind, Fordham University

The Good Data Project

Shauna Gordon-McKeon

Greenhost

GreenNet

Joseph Guay (The Do No Digital Harm Initiative)

Leslie Hawthorn

Grace Higdon

Hivos

Homo Digitalis

Human Rights Data Analysis Group

HURIDOCS

International Accountability Project

International Rehabilitation Council for Torture Victims (IRCT)

Anna Iosif

JustPeace Labs

Ken Knight

Tom Kunzler, Open State Foundation

Danny Lämmerhirt

Lebanon Support

Béatrice Leydier, gui2de

The Linkage Project

David Losada Carballo

Meedan

Connie Moon Sehat 

Convene North

Angela Oduor Lungati

Amanda O’Keefe

Powered by Data

Privacy International

Public Data Lab

R3D: Red en Defensa de los Derechos Digitales

Linda Raftree (MERL Tech)

Alexandra Robinson

David S. H. Rosenthal

JMC Schulte

Ala’a Shehabi

Signal Program, Harvard Humanitarian Initiative

Simply Secure

Romina Suarez

Berhan Taye Gemeda

tracking.exposed

Ushahidi

Patrick Vinck

Nick Wehner, Open Communications for The Ocean

Michael Wells

WeRobotics

Mushon Zer-Aviv, The Public Knowledge Workshop 

About the contributor

Jennifer's work focuses on strengthening responses to armed conflict and mass human rights abuses. As CEO and Co-Founder of JustPeace Labs, she is dedicated to creating and promoting ethical software for deployment in complex environments. She’s an attorney with over ten years of experience specializing in international justice, human rights law and peacebuilding, and someday she will finish her doctoral thesis on the law of peacebuilding. She has extensive experience working with various NGOs, universities, speaking at conferences and publishing academic articles and books. When not reading, writing and thinking about challenges to peace, she loves to practice yoga, travel, play with her sons and eat all the food.

See Jennifer's Articles

Leave a Reply


Related /

/ July 26, 2022

Takeaways from our Community Call on Responsible Data for Social Justice Organisations

/ February 15, 2022

Responsible Data Community Retrospective series: 12 articles about the state of the Responsible Data work

/ March 2, 2021

Curating connections in the Responsible Data community