Skip to content

Validating the Effectiveness of a Rapid Assessment Tool for Routine Health Information Systems

Validating the Effectiveness of a Rapid Assessment Tool for Routine Health Information SystemsThe World Health Organization (WHO), with the United States Agency for International Development (USAID)-funded MEASURE Evaluation, has identified and harmonized standards for data management of Routine Health Information Systems (RHIS). Standards, or best practices, promote the production of timely, accurate data for use in program planning, monitoring, and evaluation. An assessment tool, the RHIS Rapid Assessment Tool, has been developed to help identify gaps and weaknesses (aspects of health facility and community information systems that do not reach the identified standard) and, thus, to strengthen RHIS and improve the data. The tool compares a country RHIS to the global standards, and indicates where resources should be invested to improve the system.

The standards in the RHIS Rapid Assessment Tool were identified largely from expert meetings convened by MEASURE Evaluation and WHO in 2012–2014. In May 2012, MEASURE Evaluation hosted the International Workshop on Guidelines for Data Management Standards in Routine Health Information Systems, in Johannesburg, South Africa. RHIS experts gathered from around world to identify standards and best practices for RHIS, particularly for data management—the element of RHIS that had been identified as the most problematic globally following implementation of the Health Metrics Network (HMN) Country Assessment Tool in more than 85 countries (2008). The discussions were organized around four thematic areas: (1) user’s data and decision support needs; (2) data collection, processing, analysis, and dissemination of Information; (3) data integration and interoperability; and (4) governance of RHIS data management. The standards were field-tested in Nigeria and Bangladesh by MEASURE Evaluation in 2013–2014.

In June 2014, WHO hosted the Technical Consultation on Monitoring Results with Health Facility Information Systems, which took place in Glion-sur-Montreux, Switzerland. The workshop output was developed into a toolkit according to an organizing framework for the key components of a country health facility information system. Those components were (1) governance, an overarching component; (2) data collection and management; (3) data quality and analysis; and (4) data dissemination and use. Within each section, key action steps are identified for countries and provided with examples of available tools and resources to support country action. A checklist of key items and attributes is provided to facilitate monitoring of progress toward defined standards. The checklist is also available as a separate spreadsheet. WHO and MEASURE Evaluation decided to harmonize the two resulting lists of standards, which then became the RHIS Rapid Assessment Tool. The two source documents are available at the following links:

The RHIS Rapid Assessment Tool facilitates comparison of country RHIS with global standards, to identify gaps and weaknesses in the system as part of strategic planning for RHIS. The tool was tested in four country workshops (Madagascar, Malawi, Myanmar, and northern Syria) to validate its effectiveness. Workshop organizers then provided feedback and recommendations to help modify the tool to better meet the needs of users. The Republic of the Gambia provided feedback without testing the tool in a workshop setting.

This document describes the validation process and its results.

Monitoring Outcomes of PEPFAR Orphans and Vulnerable Children Programs in Lesotho: ASSIST Project 2017 Survey Findings

Monitoring Outcomes of PEPFAR Orphans and Vulnerable Children Programs in Lesotho: ASSIST Project 2017 Survey FindingsThe HIV epidemic in Lesotho has left one-third of the children in the country vulnerable to the virus, often without parents to care for them. Recognizing the enormous need for programs and services for orphans and vulnerable children (OVC), the United States President’s Emergency Plan for AIDS Relief (PEPFAR) has partnered with the Government of Lesotho to strengthen services for OVC and their households. PEPFAR support has focused on delivery of a comprehensive set of core interventions that include healthcare and referrals for nutrition; linkages to HIV testing, care, and treatment, including integration of adherence to antiretroviral therapy (ART) in routine household monitoring; promotion of positive parenting; economic strengthening activities for households, such as group savings and loans, cash transfers, and food subsidies; and educational support for children.

Recognizing the need to better understand the effects of its programs on the well-being of OVC, PEPFAR launched a global reporting requirement in 2014 to monitor the outcomes of selected projects in Lesotho and the other countries where it provides support for OVC. The requirement involves the collection of data for nine outcome indicators, referred to as the PEPFAR monitoring, evaluation, and reporting (MER) OVC Essential Survey Indicators (ESIs). In 2016, the United States Agency for International Development (USAID)/Lesotho requested assistance from the USAID- and PEPFAR-funded MEASURE Evaluation project to conduct a survey that would collect the required data for the Applying Science to Strengthen and Improve Systems (ASSIST) Project, its largest OVC project, implemented by University Research Co. LLC in partnership with six local implementing partners.

This report presents findings from the survey that MEASURE Evaluation, along with its local research partner, Nonyana Hoohlo and Associates, conducted for the ASSIST Project in September 2017.

Using Social Media Data to Understand Changes in Gender Norms: Guidance

Using Social Media Data to Understand Changes in Gender Norms: GuidanceAs technology becomes increasingly accessible around the globe, social media has emerged as a tool to help researchers and health workers and administrators understand a broad array of health issues. It has been harnessed to surveil infectious disease, forecast asthma exacerbations, monitor mental health following traumatic events, and gauge attitudes and beliefs about medical conditions, products, and treatments. Gender norms—the socialized expectations about women, men, boys, and girls and the power dynamics between them—constitute one frontier for social media research. Although many national and international programs seek to change harmful gender norms to achieve equities in health, few collect data on societal attitudes. Because social media provides people with an opportunity to share aspects of their lives, it may have the potential to provide insights into attitudinal and behavioral aspects of gender inequality and to capture information that is difficult and costly to obtain through regular surveys.

A MEASURE Evaluation study explored the feasibility of using large social media data sets to track changes in attitudes toward, and gender norms regarding, sexual relationships between younger women and older men and gender-based violence against women and girls in sub-Saharan Africa. After reviewing several possible social media platforms, including Facebook, Snapchat, and Instagram, we selected Twitter for its data availability and ease of access. We assessed existing methods for extracting data from Twitter and analyzing metrics to understand the challenges and limitations of using social media data. We also reviewed ethics and data security considerations with each method.

The purpose of this document is to provide guidance on collecting, analyzing, and interpreting Twitter data on gender norms. We will discuss when social media can be useful in monitoring, evaluation, and research; what data are available; and methodological challenges including generalization, biases, protecting individual privacy, and considering ethical implications.

This guidance is written for monitoring and evaluation officers or data users with some background in Microsoft Excel.

Using the Spatial Quality and Anomalies Diagnosis (SQUAD) Tool to Identify and Correct Data Anomalies in Master Facility Lists: Global Operational Guide

Using the Spatial Quality and Anomalies Diagnosis (SQUAD) Tool to Identify and Correct Data Anomalies in Master Facility Lists: Global Operational GuideA master facility list (MFL) is a catalog of health facilities that helps Ministry of Health officials know the name of each facility, where each one is located, and other important information. The MFL facilitates service delivery planning and helps locate health facilities near the populations who need them. Ensuring that the data in the MFL are correct—and identifying and troubleshooting errors—are critical but problematic, because the database has thousands of records and the use of a geographic information system (GIS) is required to check the location of each facility.

MEASURE Evaluation, funded by the United States Agency for International Development (USAID) and the United States President’s Emergency Plan for AIDS Relief (PEPFAR), created the Spatial Quality and Anomalies Diagnosis (SQUAD) Tool to assess data quality issues in these large data sets. The tool performs a data quality check on the data set, rapidly and automatically looking for anomalies in the data that need to be investigated.

This document is a resource for developing an action plan to improve data in an MFL through the use of the SQUAD Tool. It provides detailed instructions on how to prepare the data for use with the tool and how to run the SQUAD Tool with a typical country MFL. It also provides recommendations on specific steps to follow to prioritize actions and resolve errors found in the output list provided by the tool.

The status of health facilities is always changing as new health facilities come into existence and others close or move. Because of the dynamic nature of the MFL, it is recommended that the SQUAD Tool be regularly used, in line with the local MFL governance policy.

Access the resource.

Measuring Impact Qualitatively

Measuring Impact QualitativelyBased on a document review of 32 USAID-funded HIV/AIDS-related evaluation reports from the USAID Development Experience Clearinghouse (DEC) and participant research, the Measuring Impact Qualitatively paper provides insights and strategies for evaluators undertaking performance- and impact-focused evaluations. In outlining some of the ways in which integrating qualitative research methods into evaluations occurs in perfunctory ways, the paper advocates expanding discussions and moving beyond the status quo with end line performance evaluations.

The way forward, the paper suggests, centers on improving recognition that the use of qualitative research methods within evaluations cannot simply be quick and simplistic or a last-minute addition or afterthought; yet, improving qualitative research methods within evaluations is not overly complicated. To this end, the paper provides logical, common sense suggestions that emerged from reviewing 32 evaluation reports and thinking through choices, strengths, and weaknesses. The paper concludes that the endeavor to “measure impact qualitatively” is less about developing a new framework and much more about recognizing that evaluations have long sought to conceptualize, understand, and assess impact in qualitative terms. The improvements needed involve ensuring that (1) the use of qualitative research methods is thoughtful and attentive to detail, explanatory text, and transparency; and (2) the evaluation considers gender appropriately.

Access the resource.

Monitoring Outcomes of PEPFAR Orphans and Vulnerable Children Programs in Haiti: Bien Et ak Santé Timoun (BEST) 2018 Survey Findings

Monitoring Outcomes of PEPFAR Orphans and Vulnerable Children Programs in Haiti: Bien Et ak Santé Timoun (BEST) 2018 Survey FindingsThe AIDS epidemic in Haiti has left many children in the country vulnerable to HIV, often without parents to care for them. Recognizing the enormous need for programs and services for orphans and vulnerable children (OVC), the United States President’s Emergency Plan for AIDS Relief (PEPFAR) has partnered with the government of Haiti to strengthen services for OVC and their households. Since 2010, PEPFAR OVC support has included services such as HIV testing and linkages to care and treatment, potable water, immunizations, access to healthcare and psychosocial support, provision of school fees and supplies, dietary assessment and nutritional support, HIV prevention and life skills programs, and assistance with income generating activities for foster families and caregivers.

To better understand the effects of its programs on the well-being of OVC, PEPFAR launched a global reporting requirement in 2014 to monitor the outcomes of selected projects in Haiti and other countries where it provides support for OVC. The requirement involves the collection of data for nine outcome indicators, referred to as the PEPFAR monitoring, evaluation, and reporting (MER) OVC essential survey indicators (ESIs). In 2016, the United States Agency for International Development (USAID)/Haiti requested assistance from the USAID- and PEPFAR-funded MEASURE Evaluation project to conduct surveys to collect the required data for two of its ongoing OVC programs: the USAID Bien Et ak Santé Timoun (BEST) project and the Partners in Health (PIH) project, funded through the United States Centers for Disease Control and Prevention.

This report presents the findings from the survey that MEASURE Evaluation, with its local research partner Société d’Etudes et de Formation en Information Stratégique (SEFIS), conducted for the BEST project in March 2018. Survey results for the PIH project are reported here.

Caracterizando Parceiros Sexuais Masculinos de Raparigas Adolescentes e Mulheres Jovens em Moçambique

Caracterizando Parceiros Sexuais Masculinos de Raparigas Adolescentes e Mulheres Jovens em MoçambiqueMundialmente, as raparigas adolescentes e mulheres jovens (AMJ), de 15 à 24 anos, são desproporcionalmente afectadas pelo HIV/SIDA (Karim, Baxter, & Birx, 2017; Dellar, Dlamini & Karim, 2015). Em países com epidemias generalizadas do HIV na África Subsaariana, a adolescência marca um aumento na prevalência do HIV e o surgimento, e expansão dramática, das disparidades de género no HIV (Idele, et al., 2014). Estimativas recentes de sete países africanos descobriram que a prevalência do HIV entre mulheres com idades de 15 à 25 é mais do que o dobro de suas contrapartes masculinas (Brown, et al., 2018).

Apesar do mandato epidemiológico urgente e dos direitos humanos para ajudar as AMJ a permanecerem livre do SIDA, a programação nessa área tem tido um sucesso limitado. Menos da metade das AMJ que vivem com o HIV conhecem o seu status do HIV (Brown, et al., 2018), e o aumento da utilização de tratamento e as taxas de supressão viral global entre os adolescentes e os jovens, especialmente entre as mulheres, são extremamente baixas (Lamb, et al., 2014, Auld, et al., 2014, Denison, et al., 2015). Além disto, enquanto outros grupos etários tiveram declínios de mortes relacionadas com o SIDA, os adolescentes pelo contrário, tiveram as mortes relacionadas com a SIDA aumentarem em cerca de 50 por cento, entre 2005 e 2012 (Idele, et al., 2014).

Uma estratégia é prevenir que os parceiros sexuais masculinos das AMJ, que são HIV negativos, adquiram o HIV, e também reduzir a infecciosidade dos parceiros masculinos que são seropositivos. Esta estratégia requer informações sobre as características dos parceiros sexuais masculinos das AMJ, de modo que o Plano de Emergência do Presidente dos Estados Unidos para o Alívio do SIDA (PEPFAR), através da Iniciativa Determinada, Resiliente, Capacitada, Livre de SIDA, Aconselhada e Segura (DREAMS) —http: // www. dreamspartnership.org— convocou missões da Agência dos Estados Unidos para o Desenvolvimento

Internacional (USAID) no exterior para resolver esta lacuna de conhecimento através de estudos sobre as características dos parceiros sexuais masculinos das AMJ. A USAID/Moçambique solicitou a MEASURE Evaluation (Avaliação MEASURE), financiada pela USAID e pelo PEPFAR, para estudar as características de homens que se envolveram recentemente em relações sexuais com as AMJ. Este trabalho segue as linha dos esforços da MEASURE Evaluation para controlar a epidemia do HIV. O estudo resultante buscou respostas para as seguintes questões de pesquisa:

1. Quem são os parceiros sexuais das AMJ?

2. Se os comportamentos sexuais de risco entre as AMJ e os seus parceiros masculinos estão associados à certas características do parceiro sexual?

3. Em que medida os parceiros sexuais masculinos das AMJ estão usando, ou estão dispostos a usar, os serviços de HIV/AIDS?

Caracterizando Parceiros Sexuais Masculinos de Raparigas Adolescentes e Mulheres Jovens em Moçambique