From the Principles for Digital Development: Building Technology for Quality Improvement with Belize’s Ministry of Health

This post is from 2019. Some details, like features, plans and prices, may have changed since.

This is a repost. The case study below was written by TecSalud and first published in April 2019 by the Principles for Digital Development. It is reproduced here word for word under its Creative Commons Attribution-ShareAlike 4.0 license. Read the original case study (PDF).

Principle(s) Addressed: Use Open Data, Open Standards, Open Source & Open Innovation; Design with the User

Overview

In partnership with the Inter-American Development Bank (IDB) and various Central American Ministries of Health, TecSalud is designing and building a digital system to support government hospitals’ continuous quality improvement (CQI) initiatives.

The main objective of the CQI platform is to enable hospital and community health center quality teams to periodically audit prenatal, obstetric, postnatal, newborn, and child healthcare through a series of auto-calculating indicators. A randomization process prompts team members to pull patient files and score provided services and recorded data. These scores are measured against national standards of care.

The CQI platform is currently in use in four national hospitals and several regional community health care facilities across Belize. New country programs are being added in 2019.

• Objective: Audit patient files of prenatal women, postnatal women, and newborns on a monthly basis. Input data from patient files to score the service provided against a set of standards of care, to monitor continuous quality improvement.
• Where: Active in four government hospitals in Belize. More countries coming online in 2019.
• Users: Medical stuff in health centers such as administrators, nurses, supervisors, and doctors.

Background

Founded in 2017, TecSalud supports the design and implementation of health technology programs across Latin America and the Caribbean.

There are a number of commonalities in the region, from language and socioeconomic considerations to migration and health outbreaks that make working in the region both challenging and exciting. The region has experienced significant growth and investment in public health and health technology, with an increasing number of countries adopting health information systems strategies.

The 33 countries in Latin America and the Caribbean are spread farther across the development spectrum than any other region in the world. While countries like Argentina and Chile are testing augmented intelligence in hospitals, countries like Haiti and Honduras are working to bring more of their citizens online. Such differences are in part responsible for limited inter-country collaboration and the persistence of national health technology silos.

“Being that our country has limited human resources the mobile platform has enabled us to be able to collect more data in a more time efficient manner.”

— Dr. Armelle Gillett, Quality Improvement Officer, Ministry of Health

Project Lifecycle – CQI Redesign

Analyze & Plan

TecSalud applied its standard landscape analysis methodology to gather needs and pain points of the current quality improvement process in Belize. An Open Data Kit (ODK) application had already been in use for one year, along with other piloted tools that had been phased out before the project’s start. Through user feedback, technology skills and abilities evaluations and platform feature set reviews, a plan was developed to phase out the ODK platform and replace it with the CommCare platform.

The implementation plan focused primarily on developing a training and engagement strategy that could mitigate potential technology burnout and encourage managers and end users to actively participate in the redesign. The Belize team had already experienced transitioning between several data collection platforms, so it was essential for TecSalud to adapt and meet them where they were in the digital improvement process. Dr. Armelle Gillett, Quality Improvement officer for the Belize Ministry of Health, commented on her experience in the early days of moving from paper-based forms to ODK prior to TecSalud’s engagement. “The shift from the paper data collecting to using a mobile platform has been a rocky road for our team but is has also been very helpful. Being that our country has limited human resources the mobile platform has enabled us to be able collect more data in a more time efficient manner.”

Design & Develop

TecSalud requires all project partnerships to appoint one local team lead who must be present and actively participate in the tool’s design process. Unlike many digital development projects, TecSalud worked directly with the Ministry’s quality improvement officer from the beginning of the process. There was no third-party NGO involved at any stage, a major factor in TecSalud’s ability to work efficiently and gain local buy-in. Dr. Gillet shared that “ministries should work directly with Technology consultants where possible is a better approach rather than working through other organizations. This allows for better understanding of the country’s needs.”

During the initial stages, both TecSalud and the Ministry received support and technical assistance from the Inter-American Development Bank. This initial financial support and guidance was integral to to build relationships in-country.

Another key factor in TecSalud’s design phase was using a software as a service (SaaS) platform rather than creating a totally custom tool. TecSalud conducted a detailed pricing analysis to ensure that the best tool and pricing option was selected, ensuring sustainability and long-term interest in maintaining it. This was accomplished by valuating Belize’s needs by budget, technology features, and support needs, and resulted in the selection of a mid-tier pricing plan for CommCare paired with eTab, an open source data visualization tool that was first developed by the Ministry of Health in El Salvador.

The CQI platform developed includes complex algorithms, customizable indicator configurations, real-time scoring and progress reports. Adjustments to workflow rules lightened the platform significantly compared to the previous ODK version.

Deploy & Implement

Once the application contained the validated content and appropriate features, remote quality check and training milestones were met by local staff to ensure readiness for on-site training.

TecSalud’s deployment approach recommends local partners reach standard technology readiness markers to best prepare for on-site training. The local team lead and a few “super users” (select number of more engaged end users who participate in TecSalud’s quality check and readiness process) play an integral role, providing final detail feedback to be incorporated before deployment.

Periodic capacity building sessions are an important piece of TecSalud’s program implementation model, no matter the size or scope of the project. It’s for this reason that TecSalud encourages funders and project partners to invest in training. Program launch workshops, manager training and semi-regular refresher courses to address new waves of content or staff turnover are equally important.

Program Characteristics – CQI in Deployment

Monitor & Evaluate

The CQI platform is a monitoring and evaluation tool designed specifically in line with government needs. It includes a number of indicators across health care areas. Indicators can be broken down into a number of criterion that contribute to the calculation of the score for whether a particular activity is meets quality improvement standards.

For example, a doctor may have a number of, let’s say, 10 steps that she or he must record to complete a prenatal visit that meets the prenatal care national quality improvement standard. In order for that doctor’s visit to receive a passing score, the CQI platform calculates which were completed and whether they completed in the right way. That number is measured against the total and scored based on the number out of 10 that must be completed to receive a “meets standard” score. If the passing standard is 8 out of 10 and the doctor has received a 7, that patient visit receives a “needs improvement” score.

Over 20 indicators are currently configured for continuous monitoring and evaluation. Health care teams input visit information into the CQI platform monthly and quarterly. The information is pulled directly from patient files and individual scores are calculated per indicator in real time.

Individual scores roll up and are aggregated to then be sent to the eTab data dashboard. Longitudinal visualizations show progress made from month to month against quality standards. Filtering options allow facilities to see their individual progress compared to other facilities.

In conjunction with the Ministry of Health in Honduras, TecSalud is scaling this continuous quality improvement platform to 84 health facilities in Honduras. A follow-up to this case study will be released in the summer of 2019.

Opportunities and Challenges

• Language and terminology are important. The previous quality improvement platform featured screens that would tell the user they “failed” if their score fell below the threshold. During interviews with users, they shared that they found this to be demotivating and even hurtful. The rigidity can be softened by incorporating slight tweaks to terminology that sit better with users.
• Using a preexisting platform allows for greater flexibility and user involvement during program design phase. In terms of scope, this can be tricky to manage but if using a platform that allows for quick changes to details like wording there’s a significant opportunity to engage with end users. By simply demonstrating a suggested (and validated!) change to wording has been made while sitting with the individual who made the suggestion, he or she may feel significantly more engaged with the process.
• Consolidating change requests and explaining how they are done is critical. Facilities using the CQI platform are far from one another and don’t always follow the same quality improvement processes. Requests to change content and details of the platform are collected by one individual in each region and roll up into a change request form shared with TecSalud. This enables TecSalud to incorporate changes on a monthly basis, not on a rolling basis. This process has helped tremendously in the adoption of the tool and to overcome the challenges related to local users understanding the configuration is for national use, and not every detail may exactly fit across facilities. It has significantly reduced technical problems related to application versioning.
• Don’t offer minimal improvement — create significant changes. Using the principle, Use of Open Standards, Open Data, Open Source, and Open Innovation has allowed us to expand what was already created. This increased data quality while also decreasing the amount of time it takes the user to work with the tool. Even a minute saved for the user is a significant change.
• The principle, Design with the User, encouraged us to be cognizant of the stage your users are at. When designing digital solutions with end users, there are different considerations when transitioning between paper and digital vs. when you are replacing one digital tool with another. In this case, the users had already passed the process of moving from paper to digital. In order to secure user buy-in, TecSalud had to ensure that the replacement digital solution was significantly (not marginally) better than the ODK predecessor. To learn more about the Principles of Digital Development, please visit: digitalprinciples.org

Resources

• El Pais article about the work in Belize
• Review of 3 popular digital data collection platforms (TecSalud blog, no longer online)
• Resources for investing in digital tools presentation (TecSalud blog, no longer online)
• Digital Development in Latin America and the Caribbean (TecSalud blog, no longer online)

“Case Study: TecSalud. Building Technology for Quality Improvement with Belize’s Ministry of Health” by TecSalud for the Principles for Digital Development, licensed under CC BY-SA 4.0. Reposted without changes to the text; section layout and photo captions added for the web. This repost is shared under the same license.