Planning for compasses, not maps: BID celebrates a culture of learning, applies principles of adaptive management
By Celina Kareiva, Communications Associate, BID Initiative
Mar 16, 2018
Posted in Monitoring & Evaluation, People, Policies & Practices, Practices, Products
This is the second blog in a two-part series about adaptive management, including lessons from the BID Initiative.
Beatrice Owawa’s fingers dance across the screen of her tablet. She nimbly jumps from one window to the next, as she checks vaccine records and stock levels in the Tanzania Immunization Registry (TImR) at Usa River Health Center, where she works as a medical attendant. When Beatrice pulls up a list of young patients who have recently missed their vaccines, she takes note of their names and contact information and begins to reach out to their caregivers.
It may seem unremarkable, but the task of identifying patients who have fallen through the cracks used to be nearly impossible. Before TImR, Beatrice relied on paper registries to track “defaulters,” or patients who had missed a life-saving vaccine. The sheer volume of patient information – stored in immunization registries several inches thick – made it difficult to track a patient’s entire history through data, let alone ascribe any meaning to that data. It meant that a patient who had missed a vaccine went unnoticed and unprotected.
With TImR that process is now automated. Beatrice can pull up a list of defaulters and their contact information instantaneously. Her swift use of the system to track down patients who are overdue for vaccines illustrates the power of adaptive management, a set of tools, processes, and approaches quickly gaining momentum in the global health field that calls for a new way of working in complex and changing environments.
Instead of enforcing a set of static practices or methodologies established before the start of a project, adaptive management embraces strategies, decision-making processes and timelines that allow for iteration in the face of new knowledge, new circumstances, or new needs. Adaptive management has implications both at a health systems level, for how health workers respond to and act on – for instance – patient data, and for program implementers, as they flex intervention strategies when new developments come to light. At a micro-level, it may entail the use of participatory facilitation techniques to invite alternative perspectives during team meetings; and at macro-level it may involve structured “time-outs” to reflect on program direction. At its heart is a continuous process of “learning by doing” and steady improvement.
Pivoting approaches in the face of new information
Learn fast, fail fast, share fast has been a core value of BID from the earliest days. BID didn’t predefine solutions or demonstration countries; instead it partnered with countries to identify the most critical routine immunization service delivery problems. In embracing this philosophy, BID has created an enabling environment for adaptive management and a commitment to iterating that breaks from the risk-adverse culture that characterizes most development programming.
It can be seen in the establishment of User Advisory Groups, which serve as learning channels that center the voices of health workers, and in BID’s strong feedback loops between program staff and stakeholders. At a micro-level, adaptive management plays out in the form of weekly meetings to review progress and a culture that celebrates critical thinking over stringent metrics.
BID is also applying principles of adaptive management at a programmatic level. Using a series of “touches” or visits in either country, BID staff initially led on-the-job training to health workers. But the time-consuming strategy took between two to four months to implement and allowed for rollout in only one district at a time. Health workers passively participated in the trainings, and felt little incentive to use the system. As rollout accelerated and BID worked to scale interventions in either country, the training method also became increasingly expensive.
Realizing that this method wasn’t sustainable, BID shifted to a training strategy led by district authorities, or district data use mentors (DDUMs). DDUMs provided prompt and localized support, and reinforced behavior change by allowing health workers to see their peers using the system. Rather than forcing a method that had garnered little traction, BID evolved its training to meet health worker needs.
Similarly, when BID learned that the timing between touches didn’t align with the realities on the ground, it course corrected. Initially, the duration between trainings was based on the size of the facility’s immunization program, or whether the facility had a high or low-volume of patients. BID adjusted its timeline as it improved its understanding of nurse workflows and the technical support required by health workers. The time between Touch 2 and 3, for instance, was shortened to a week, as BID learned that nurses required prompt follow-up after receiving the new data entry tools.
Leveraging BID’s learnings
As PATH’s Data Use Partnership (DUP) builds on BID’s foundational work in Tanzania to develop a comprehensive and enabling environment for using data for primary health care decisions in the country, adaptive management will play a critical role in implementation. DUP applies iterative decision-making cycles throughout – within its monitoring, evaluation and learning (MEL) plan, agile software development, and learning-based rollout strategies.
For instance, to capture the various stages of behavior change behind data use, DUP will evolve its learning questions, monitoring and evaluation (M&E) plan, and indicators throughout. As the timeline, interventions and activities shift, certain metrics may be activated or deactivated to orient M&E functions toward programmatic and strategic questions, rather than reporting. Initially DUP may measure how many times health workers log into a new registry – a simple way to quantify system usage. With greater demand for data, the M&E plan may shed process indicators to focus on data quality and outcomes. How many times health workers use data to inform actions, for example, may provide a more accurate measure of the system’s long-term success. Qualitative measures will also help illuminate contextual influences, informing programmatic pivots based on what the data reveals. This requires abandoning some degree of rigor, to facilitate faster feedback.
“Shifting away from a rigid, traditional MEL plan to one that adapts and pivots with the implementation of DUP, allows us to be more agile and provide all of the necessary stakeholders and implementers with information they need to make informed decisions and take action,” explains Christina Bernadotte, a Senior MEL Officer for DUP. “It is a little unnerving to give up some of the rigor that comes from traditional M&E and we know this means that our measurement needs will change constantly, preventing us from looking at some trends over time, but overall we believe it will allow us to have more meaningful impact over the project’s lifecycle.”
DUP, taking a cue from BID’s culture of learning, will also build systematic periods to pause and reflect into day-to-day operations. Weekly team meetings will create space to share lessons and habitually review relevant data that are available depending on implementation priorities. A monthly learning meeting will allow for broader dissemination among PATH staff and quarterly meetings with DUP government, donors and other stakeholders will allow for joint learning, informed decision-making and adaptive management based on the most up-to-date information available to everyone. To catalogue this knowledge, technical teams will document this growth in a learning log.
As BID, DUP and other development partners apply principles of adaptive management, they usher in a new era of programming in global health. Increasingly, NGOs, donors and international organizations recognize that the most challenging problems require iterative solutions.
The benefits of adaptive management extend far beyond program teams and implementers. With access to real-time data, health workers like Beatrice are able to course correct when presented with new information. And that may mean one less unvaccinated child.
Keep up to date with the latest BLN content and blog posts.