The Role of Paper to Digital Data Solutions
By Rachel Powers, Associate, Information Systems, VillageReach
Jan 6, 2016
Posted in People, Policies & Practices, Products
A version of this post originally appeared on the Village Reach blog.
Some of the many paper registers used at one health clinic in Malawi. Photo: VillageReach/Rachel Powers.
In response to this article by Dr. Mahad Ibrahim on the role of data during the Ebola outbreak, some interesting questions were sparked among our team. The piece evoked a lot of great insights about the opportunities and challenges the piece addresses, as well as how we see those topics evidenced in our VillageReach work. This topic is of particular interest to us, as we and our partners at the University of Washington are actively developing and testing an open-source tool for paper to digital data conversion, ODK Scan. ODK Scan is an Android app that uses a smartphone’s camera and custom image recognition processing to automatically digitize information from paper forms. Our work with ODK Scan and our experience with digital tools in other programs has informed much of our understanding of this paper to digital question. Below are some general themes that regularly surface in our discussions, plus some specific examples from our experience.
Direct to digital tools often prove challenging in actual field use
The challenges that Dr. Ibrahim describes in direct-to-digital data collection are absolutely supported by our experience with various projects in this space, particularly in regards to duplication of data entry, technology troubleshooting, and lack of consistent connectivity to the network. These are big reasons why we’ve invested in researching, testing and even helping to develop paper-to-digital alternatives. (e.g., ODK Scan) One quote that strongly resonated with our staff and our experience was:
“People have established workflows and habits, which can be difficult to change. Even with mobile devices, health workers would still fill out paper forms because part of the medical system’s reporting procedures required it. What mobile devices did, in effect, was provide more work and another piece of equipment to juggle.”
Some specific examples we’ve seen of this from our work:
- Using ODK Collect for an evaluation of medicines availability at the health center level in Malawi, we initially thought that directly entering the data in digital format would be easier and quicker for data collection. Instead, we found the opposite. The data collection team had significant challenges with the linear nature of data collection using this tool (answer question 1, then 2, then 3). The ability to easily “skip around” when using paper forms is a benefit that is often lost with digital tools. While there is a potential benefit to enforcing a specific workflow, it also limited the flexibility and significantly changes work processes in a way that is unacceptable to most users.
A community health worker in Malawi conducts a home visit with an expectant mother. Using an app on her mobile phone, she fills in health information, in addition to completing a paper register. Photo: VillageReach/Rachel Powers.
Even surveyors that were highly computer and mobile device literate and experienced in data collection reverted to using paper, entering into ODK Collect in the evening. Furthermore, there were some digital-only records early on (before the duplicative form + app process) that were lost over the 3G network sync with the cloud server. This necessitated the surveyor to return to the facility and re-collect the same data, causing some collection delays and increasing frustration with the system. With a hardcopy paper record this digital data loss is far less calamitous.
- Direct-to-Digital with community health workers (CHWs): We had the unique opportunity to test both direct-to-digital and paper-to-digital processes with a group of CHWs in Malawi. Not only did the direct to digital app prove buggy (it kept crashing) and challenging for training, the phones themselves were often unusable because of a lack of charging locations. The biggest benefit the CHWs experienced from the implementation was the ability to communicate with their peers via WhatsApp about both professional and personal topics. So while the CHWs reported a predilection for having the phones, when it came to the actual data collection process they preferred the reliability of the paper registers. Furthermore, when a technology continues to fail (and not due to user error), it’s difficult to maintain user trust that a new version or new software will work.
Paper to Digital tools aren’t a magic bullet
While the article does a good job outlining many of the issues with direct to digital data collection, it doesn’t go into the challenges that still exist with paper to digital solutions which we have seen in our work.
- Paper to digital conversion tools still require maintenance or hardware and software, just like the direct to digital tools. Ideally you can cluster these in a few key locations where support is available, but it still can be a constraint. The noted Captricity solution requires a digital camera or scanner to capture images of the forms, which must then be transferred onto a computer with an Internet connection to upload the images to the cloud for processing. It’s true that these software maintenance and training costs would be much lower than direct to digital, especially if they are housed only at a district level, but it is inaccurate to portray the solution as completely free from the challenges facing direct-to-digital technologies.
- Cloud-based paper to digital conversion tools require data upload, which can be costly and time consuming in low-bandwidth scenarios. The proposed alternative of a service like Captricity still necessitates a network connection (via Dropbox), since the scans of the paper records need to be uploaded to the cloud for processing. The current paper to digital solutions available do rely on Internet connectivity at some point in the pipeline, whether it be through using engines on the cloud for data processing or simply for sending processed data to the next decision-maker. Even ODK Scan, which does in-device processing without a network connection, needs 3G/wifi connectivity for syncing with cloud-based data aggregation tools. There are commercial solutions that can perform high-powered processing on a desktop, but that still poses the problem of either getting the forms to the computer setup or maintaining a desktop system at the last mile and still needing to share the digital records upwards for aggregation. Internet connectivity is going to play SOME role in the paper to digital solution, but the correct solution will be designed in a way that is flexible to inconsistencies in connectivity (3G, specifically). Redesign of paper forms to minimize amount of data needing to be sent for aggregation will also play a big role in optimizing this process.
- There is a dearth of quality data on what tools are best suited to various paper to digital data conversion scenarios, and what technologies are possibilities for scale at a national level without requiring register redesign. We have conducted a cursory landscape, included in our ODK Scan Recommendations Report, but further examples and case studies are lacking.
- Lastly, changing official, government paper forms and registers into a format which can be easily digitally processed is not a small task. Particularly with processing numbers and letters, forms that enable more compartmentalized recording of data (e.g., one box per digit in a number) are needed for reliable processing. From our field tests of ODK Scan in Malawi, we’ve found that getting users to fill out a redesigned form comes with some challenges, which we’ve outlined in our ODK Scan Malawi final field test report. In addition, getting official approval from the Ministry of Health for redesigned forms is a long process with significant buy-in needed across a number of departments. While field testing/proof of concept initiatives with a redesigned form is possible, until official forms can be redesigned to enable automatic processing, we will continue to run into the aforementioned issues of duplication of efforts on multiple forms. It’s worth noting that while Captricity does not require form redesign, many of the other paper-to-digital options do require some form adjustments.
Identifying and developing the most appropriate system solutions for last mile data collection is certainly a compelling challenge, and we look forward to continued discussions and collaborations to address this need. Additionally, if anyone has questions about our work with ODK Scan or would like more information on our program please feel free to reach out to me directly.
Many thanks to my VillageReach colleagues who contributed to this blog: Sarah Jackson, Jodi-Ann Burey, and Jessica Crawford.