The Biggest Risk Isn’t That AI Replaces Evaluators
Reflections from Tanzania MEL Week 2026
By Jennifer Nyakinya, Director of Programs, NIERA · September 2026
On 24th September in Arusha, I had the privilege of joining a panel at the 5th Tanzania MEL Week on “Strengthening MEL Systems, AI, and Machine Learning.”
I shared the stage with Ms. Caroline Makuvire, Director, Evaluation, Research and Learning at the Office of the President and Cabinet, Zimbabwe; Dr. Awour Ponge, former Vice-President of the African Evaluation Association (AfrEA); and Prof. Provident Dimoso, Deputy Director, Academics, Research and Consultancy. What struck me most is that nobody was debating whether AI belongs in MERL. The real question on the table was how we make sure it serves the people evaluation is meant to serve.
AI is a tool, not the evaluator
I was asked how NIERA ensures that greater use of AI does not remove the human element from evaluation and learning. My starting point: AI is a tool, not the evaluator.
It can make evaluation faster. It can process large amounts of information, analyze it quickly and automate routine tasks, and that is genuinely valuable. But faster doesn’t necessarily mean better. AI doesn’t have an internal doubt meter. It can give you an answer very confidently even when the answer is wrong, and it doesn’t know, the way a human does, when to say “wait, this doesn’t make sense.”
That is why humans still need to reason more than ever. And there is a second question: whose knowledge sits inside the tool? AI systems are commonly trained on data and knowledge shaped by the Global North. That isn’t inappropriate in itself, but it means we have to contextualize the responses we get.
Our approach at NIERA is to retain community-centered approaches and strong human oversight. This in turn supports quality assurance, bias identification and, most importantly, the localization of evaluation.
This is the thinking behind the NIERA Impact Academy we are designing: a six-month hybrid practitioner program that pairs AI and machine learning skills with ethical AI use and participatory design methods, to grow homegrown evaluation leadership.
Language, trust and capacity: what my fellow panelists added
Two contributions from the panel have stayed with me.
Prof. Dimoso spoke about building local languages into AI tools and about focusing on capacity strengthening within institutions. This is localization in practice, and it connects directly to the Global North point above: a tool that can’t work in the languages of the communities we evaluate can’t truly listen to them, and institutions, not vendors, need to hold the skills to use these tools well.
Dr. Ponge highlighted the importance of lived experience, indigenous communities and building trust within communities. Trust is not something a model can generate. It is built over time, in relationship, and no dataset substitutes for it.
Build the capacity to question the tools
The closing question asked each of us for just one recommendation for governments, universities, development partners and practitioners. Mine: invest in AI literacy and responsible AI capacity, not just in types of AI tools.
Organizations are already acquiring tools, but acquiring tools doesn’t necessarily mean building capacity. Capacity means knowing when to use AI, how to interrogate its outputs, how to protect sensitive data, and when human judgment needs to take precedence. Let’s start building it now, and institutionalize it within African organizations.
The biggest risk is not that AI will replace evaluators. The biggest risk is that we will use very powerful tools without having built the capacity to question them.
A big thank you to the Prime Minister’s Office (PMO-PPCPD) and the Tanzania Evaluation Association (TanEA) for convening such a rich week, to our moderator, Taku Chirau, PhD, for steering a rich and thoughtful conversation, to my fellow panelists for the exchange, and to the NIERA members who showed up across the week, including at our masterclass, From Data to Decisions, and our exhibition booth.
For the full story of NIERA’s masterclass and our time at the exhibition booth – including how we put the Credible, Timely, Usable framework from the masterclass into practice, and the Deputy Speaker of the National Assembly’s visit to our booth – read “ NIERA’s Week at the 5th Tanzania MEL Conference” (See companion article.)