CARHAP M&E Tools

CARHAP is using the following M&E tools  for monitoring outputs and activities where external data is not available, inadequate or insufficient. This specifically involves the monitoring of improved quality and comprehensiveness of harm reduction service delivery (Output 1) and the increased capacity for providing quality harm reduction services (Output 2).

Monitoring Harm-Reduction Service Delivery (Key Output 1)

Harm-reduction (HR) service delivery (Output 1) will be monitored through a series of three linked forms that will collect key data on clients, services delivered, and trends regarding safer behaviours.

(1a) The Client Registration Form is used to register clients during their first contact with the HR-service outlet. It will also be used to assign an anonymous Unique Identifier Code (UIC). It will collect the following sets of data:

(a)   Personal client characteristics (anonymous);

(b)   Reason for visit, and Services provided; and

(c)   Key data on risk behaviours.

(1b) On each subsequent visit, the Client Contact Form is collecting data on service delivery, using the same questions (b) as in the initial Client Registration Form. This allows monitoring the specific services provided to each client, as well as the overall volume and scope of services of particular HR service-delivery sites.

(2) Every six months, the Client Follow-up Form will be administered to all clients in a defined period, in order to establish trends in risky and safer behaviours, using the same questions (c) as the Client registration Form. This periodic data collection will allow monitoring changes in behaviour and linking them to HR services provided.

Compatibility with existing M&E practices and tools

The three interlinked client-contact forms will build as much as possible on existing (best) practices among the different HR service providers in the three countries. For this purpose, a series of consultative roundtable meetings will be held in the three countries to discuss and agree on a common format for monitoring HR services in the context of CARHAP.

Furthermore, the behavioural component of the forms (Client Follow-Up forms) will be based on a selection of the key questions used by national governments (with technical assistance from CDC Almaty) for second-generation surveillance (SGS) purposes.

This unified series of forms – based on existing formats and compatible with existing national data sets (e.g. SGS) – will also form the basis for promoting one common standard for monitoring and evaluating interventions and services in the field of harm reduction, using the UIC.

Positive experiences and lessons learned in the context of CARHAP will allow the further use of these formats by other HR service providers, thus contributing to the achievement of CARHAP Output 3.

Monitoring the Capacity of HRSOs to Provide Quality Services (Key Output 2)

In addition to the forms mentioned, which allow monitoring the scope and volume of HR services, as well as trends toward safer behaviours, quality of services is a crucial aspect and programme output to be monitored. It is directly related to CARHAP’s key Output area 2, which aims to develop increased capacity for providing quality HR services.

As mentioned above, two M&E tools will address the issues of capacity and quality:

• Organisational Capacity and Quality-of-Services Tool (3) (Objective assessment of organisational development; quality and comprehensiveness of services)
• Client Satisfaction Assessment Tool (4) (Subjective assessment quality of services)

Both tools will be administered by external experts to minimise bias. In addition, however, HRSOs will be asked to monitor themselves on a regular basis, using an Organisational Self-Assessment tool. As part of the grant process (during proposal-writing workshops in April-May 2006), HRSO candidates will first be asked to complete this self-assessment form regarding their organisational capacity. This is meant to be an indicative baseline.

(3) The Organisational Capacity and Quality-of-Services (OC-QS) Tool will combine two major areas:

(a)   The Organisational Capacity (OC) of HRSOs to deliver quality services;

(b)   The actual Quality and Comprehensiveness of HR Services (QS) delivered by the HRSO.

The OC-QS Tool will be used to monitor the impact of capacity-building efforts through CARHAP, as well as the resulting quality and comprehensiveness of HR services provided. The combination of these two aspects in one tool enhances the efficient use of limited time and resources, and minimises the “M&E burden” on HRSOs.

The tool will be implemented by independent consultants from the CARHAP TA pool. The first external assessment (1) will take place within a 3-6 month period after HRSOs have started implementing their grants; this period will allow them to build some initial capacity, and avoid overburdening the grantees with additional M&E tasks. Subsequent OC-QS assessments will be conducted on an annual basis to monitor progress: (2) at mid-term; and (3) at the end of the CARHAP programme.

(4) Client Satisfaction Tool

The Client Satisfaction Tool aims to complement the previous tool with a more subjective assessment of the quality of HR services by clients themselves. It will address the same aspects of quality and comprehensiveness of HR services as used in the previous tool (3).

To minimise the M&E burden on HRSOs, the Client Satisfaction Tool will be administered in combination with the previous OC-QS tool by external consultants from CARHAP’s TA pool (3 times in the course of CARHAP).

Operational Research

In addition to the programme M&E strategies described above, Operational Research (OR) will allow addressing gaps in knowledge and understanding of key issues and factors that may affect programme implementation. This includes the impact of gender, social, cultural, behavioural and economic factors on drug use, sex work and HIV infection, as well as issues directly related to the implementation of HR programmes. The main purpose of operations research is to gain a better understanding of these issues to allow improving and/or strengthening programmes and services that are better tailored to the needs of groups most at risk. Therefore, active participation of key beneficiaries and most-at-risk groups will be given priority attention throughout the OR process.

CARHAP grant guidelines and criteria will encourage harm reduction service organizations to include a small Operational Research (OR) component in their project proposals. In addition, each country has a limited special budget for studies and assessments. In this context, a first rapid assessment of OR needs and priorities will be conducted among HRSO grant candidates during a proposal-writing workshop.

Later on in 2007, after HRSOs have started implementing programme activities and services, specific OR priorities will emerge from the implementation process, national implementing partners (NIPs) and HRSOs in each programme country will conduct National Round tables to identify joint OR priorities and a common research agenda.