Skip to content

Building the Capacity of Capacity Building

by on December 26, 2012

Guest post by Jack Hazerjian, Research Associate/Capacity Building with MEASURE Evaluation

In more recent years, there has been a growing interest within and outside of MEASURE Evaluation to adopt a broader approach in building the capacity of partners in health programming. Historically, to help improve their performance in planning, implementing, and overseeing delivery of health services, these partners would receive training, technical assistance, and better tools for work documentation and assessment. Indeed, capacity was directly equated with having the requisite resources and abilities to perform at levels allowing for pre-set objectives to be met.

However, despite such efforts and good intentions, the level of success and of sustainability in partner performance has been far from guaranteed. With newer thinking, there has been a stronger focus on reframing this process so that partners are more proactive. As such, organizational development has become more commonly used as the key strategy, by which efforts are placed in improving an organization’s effectiveness in managing its own efforts to achieve performance objectives. Among interventions supportive of this has been the provision of leadership training for staff as well as for senior management.

Mindful of these developments, the Capacity Building and Training (CBT) Unit of MEASURE Evaluation recently designed a tool for an organization to self-assess its capacity in carrying out functions in monitoring and evaluation (M&E) across all its operations, programmatic as well as administrative.

“SCORE ME” (“Self-assessment of Capacity regarding Organization’s REsponse in M&E”) asks an organization to evaluate itself across eight functional areas related to good M&E practice, with about 14 performance elements in each. Corroboration of a positive rating is required in the form of documented evidence. A job aid has been created to help guide self-administration of this assessment, and there is a companion document with which the organization can plan its self-improvement on failing performance elements. Also available is a parallel set of documents for self-assessment of individual M&E capacity.

For its first field testing, Jack Hazerjian of MEASURE Evaluation’s CBT Unit introduced “SCORE ME” to the Southern Nations, Nationalities, and Peoples (SNNP) Region of southern Ethiopia. He was invited to come in late November by Tariq Azim, who works for John Snow, Inc. as the Chief of Party of the Reformed Health Management Information System Scale-Up Project. It should be noted that this is one of MEASURE Evaluation’s most intensive project efforts during Phase III, with an improved health services data management and transmission system established at more than 2,500 locations within that region and with more than 6,300 individuals trained.

Provided administrative support by Azim and his staff based at the regional capital of Awassa, Hazerjian led introductory sessions with two groups of M&E officers: one from the regional health bureau and another from a sub-regional health department. After receiving a few hours’ instruction, the two groups were able to conduct a self-assessment of their organization’s capacity in M&E with minimal support. Each focused its review on two of the eight functional areas in the instrument — those specifically addressing health programming M&E. In respect for the self-guided approach that is sought from organizations that use “SCORE ME,” the selection of functional areas is left for them to choose.

The results from their self-appraisals and self-improvement plans were presented in a wrap-up session, with the regional health bureau director in attendance.

Summary findings from the presenters on M&E organizational capacity:

  • Both those from the regional health bureau and from the sub-regional health departments felt that overall competency in program monitoring was stronger than in program evaluation, with each citing more than two dozen monitoring-related documents as evidence of institutionalized good practice
  • Interestingly, each noted one, but different, element as a deficiency in their respective program monitoring performance: data analysis, data quality
  • As for program evaluation capacity, each gave their work department low marks, citing weaknesses in abilities to evaluate effectiveness and efficiency in implementing health programs as had been planned and in meeting pre-set targets
  • Both noted lack of sufficient resources for conducting quality assessments of work performance and work data
  • Both noted the need for training in evaluation methodologies, specifying in particular the development of an evaluation framework

In response to the presentations, both the regional health bureau director and Azim noted that health administrators in the SNNP Region are reasonably competent in program planning, although their success in program implementation is admittedly spotty. Azim cautioned against the “automatic” response that additional training is necessary to address this, although he supported the need to look into underlying causes. He thought that the use of “SCORE ME” to self-assess implementation planning might help serve this purpose, and Hazerjian will follow up on this suggestion.

From this first testing of “SCORE ME,” Hazerjian learned from the example of the M&E Officers from Ethiopia’s SNNP Region that the instrument did not require more than one day’s instruction for self-administered use. However, he recognized the need to simplify some of the text in the documents, especially the job aid and the evaluation section of the self-assessment instrument on organizational capacity. Meanwhile, the individual self-assessment instrument was reported to be relatively self-explanatory. In short, he felt that “SCORE ME” was moving closer to the target of having MEASURE Evaluation’s project partners taking greater ownership in the process of capacity building.

Review MEASURE Evaluation’s sets of documents for self-assessment of organizational and individual capacity in M&E.

From → Uncategorized

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: