LEADERSHIP IN MONITORING

What are, in general, the objectives of monitoring and evaluation?

Important objectives of reflexive monitoring are to learn from practical experience and to apply the findings in a project or programme for system innovation. But there are also other objectives of monitoring and evaluation that you may have to meet. The following list presents the different objectives and their consequences for the use of the method and the target groups.

Objective: learning

To meet this objective you collect information about the progress of your project or programme, about the intended or unintended effects, about developments in the environment and about obstacles that you encounter. You use this information to reflect on your points of departure and your assumptions about the problem or about the possible solutions or the knowledge needed to find solutions. This might lead to the conclusion that you have to revise your automatic assumptions.

Target group: In terms of learning, an important target group are those who are directly involved in carrying out the project or programme. Other important target groups are people engaged in related experiments, managers and financiers. Policymakers should be involved in the process, particularly when reflecting on institutional barriers.

Objective: accountability

The question of accountability mainly concerns whether you have achieved the goals that have been set and have made effective use of the resources provided.

Target group: In this case, the target group is usually the client or financier of a project or programme. In the case of national, government-financed programmes it is the government and parliament.

Objective: intervention or adaptation

Here the objective of the monitoring and evaluation is to decide whether the transition project or programme needs to be modified, for example in terms of the activities or target groups. This step is an integral part of reflexive monitoring

Target group: Where adaptation is concerned, the most important target groups in reflexive monitoring are stakeholders and the programme management. But interventions or adaptations can also arise from the conclusions reached by clients/financiers as to whether the money has been properly spent, in which case the client and the programme management are the most important target groups.

Objective: generating enthusiasm and transferring knowledge

Creating enthusiasm and transferring knowledge can contribute to the legitimacy of system innovations and help increase support for anchoring them in an organisation.

Target group: The target group consists of the client/financiers and potential or existing fellow innovators, possible future financiers and insurers, regulators or a wider public.

Key Principles For Monitoring And Evaluation

• Monitoring and evaluation should refer back to existing baseline data or begin with a baseline study
• Monitoring and evaluation are closely linked to, and should reflect the methodology of, the initial project appraisal
• Monitoring is an integral part of programme implementation
• Monitoring should be linked to the specific programme objectives that were defined through the appraisal and programme planning process
• Monitoring information should be used in decision-making
• Evaluation should follow a specific methodology designed to gather information about programme success
• Monitoring and evaluation should respect principles of participation and involve all programme stakeholders, including programme implementers, beneficiaries (programme users), women and girl survivors when appropriate and safe, local officials, and other observers such as related civil society groups
• Monitoring and evaluation should respect and protect the rights, welfare, and confidentiality of all those involved in the programme

When should Monitoring Take Place:

1. M&E is a continuous process that occurs throughout the life of a program (PCM).
2. To be most effective, M&E should be planned at the design stage of a program, with all the resources (time, money, and personnel) that will be required calculated and allocated in advance.
3. Monitoring should be conducted at every stage of the program, with data collected, analyzed, and used on a continuous basis. 
4. Usually about 7% of the total budget of the project is allocated to M&E
5. Evaluations are usually conducted at the end of programs. However, they should be planned for at the start because they rely on data collected throughout the program, with baseline data being especially important. 

What to monitor

Level in Objective Hierarchy	What to Monitor  and Evaluate
Activities	Have planned activities been completed on time and within budget? What unplanned activities have been completed.
Outputs	What direct tangible products or services has the project delivered as a result of activities.
Outcomes	What changes have occurred as a result of the outputs and to what extent are these likely to contribute towards the project propose and desired impact.
Impact	To what extant has the project contributed towards its longer terms goals? Why or why not? What unanticipated positive or negative consequences did the project have? Why did they arise?

Types of Monitoring

1. Process Monitoring  (Real Time Monitoring)
2. Progress Tracking
3. Progress Validation
4. Performance Monitoring

Process Monitoring:

Process monitoring is a key component of any M&E system. Process monitoring informs management and a donor about the actual implementation of project activities in the field. At the same time process monitoring let the project staff on ground know how well they implements the project and what improvement they can bring to the work they are doing in field.

Process monitoring is conducted using checklists and guidelines. Those checklists are developed jointly with project staff. The same checklists and guidelines are used by field staff while implementing project activities. Following the same checklists/guidelines by both the monitoring staff and the field staff help the M&E staff to identify and share gaps that are identified during the process monitoring. Participants were shared a sample of monitoring guidelines. In order to undertake process monitoring, a monitoring tool is required that capture the following information:

1. Purpose of the monitoring visit
2. Which activity does the visit covers
3. Methodology adopted for the visit
4. Key findings from the field
5. Feedback by the field staff
6. Debriefing points agreed
7. Deadlines and responsibilities

A sample tool for field level process monitoring was shared with participants. Participants were oriented on rating of process monitoring reports.  It is important to quantify monitoring findings for better analysis. Traffic lights (Green, Amber, and Green) are usually applied for rating findings from the monitoring visits. The lights are explained below:

Progress Tracking

Every project has set output targets to be achieved. One of the key functions of any robust M&E system is to capture progress against those output targets. A tracking sheet is required to outline all output indicators for key activities along with target values for those output indicators. The targets could be divided into quarters/years. Progress is entered against those targets and the trackers automatically calculate deviation against the targets.

Progress is tracked for two reasons:

To see whether the project is on-track or off-track

1. To assess whether time-critical activities are taking place as per the calendar or not.

The information could be used by partners in writing and submitting progress reports to donors.

Progress Validation:

Progress validation is another important type of monitoring. Progress of key project activities are usually reported by the field staff. In order to validate the output progress reported, the M&E staff collect the Output Tracker and identify output indicators to be validated. Validation/verification is initiated by collecting of Means of Verification (MoV). Once MoV are collected, the M&E staff takes a sample out of those and physically verifies the activities. This is followed by assessing and verifying thoroughly the process being adopted by field staff while conducting that specific activity. At the same time, if the activity has been undertaken a while ago, performance and outcomes of the intervention is also assessed.

Output is validated in field using the same tool and guidelines used for process monitoring.

Performance Monitoring:

Participants were explained that projects are mainly designed and funded to achieve desired outcomes. Assessing those outcomes and changes are the key functions of M&E Unit. ‘Value for Money’ of a project is assessed through assessment of performance indicators.

Performance or outcome indicators are usually outlined from the project proposals and these are inserted into an M&E plan.

To assess progress for performance indicators, baseline is important. Baseline data shows the pre-project status of performance/outcome indicators. There are certain indicators which associated with behavior change. For those indicators KAP studies (Knowledge, Attitude and Practices’ are conducted. Baselines/KAP studies are conducted through systematic process and methodologies. Once baseline data is collected, the values are inserted into the M&E plan. Based on the baseline data, realistic targets are set together with the project implementing staff. Once targets for the outcome/performance indicators are set, sources of data and methods of data collection for the indicator is identified.  This enables the M&E staff to be aware of data collection sources and sampling to be followed. The M&E plan also provides information about frequency/timeline for each performance indicator to be assessed over a period of time.

Periodic assessments are conducted using the same methodology and tools of the baseline to track performance indicators. Like other M&E studies, periodic assessments are initiated by drafting a Concept Note. The Concept Note briefly outlines:

1. Purpose of the assessment 
2. Scope and indicators to be tracked
3. Methodology, tools and sampling size
4. Type of tools to be used for the data collection
5. Who will participate in the data collection
6. When and where the data will be collected
7. How the data will be managed and analyzed
8. Reporting and timeline

Once periodic assessments are completed, the analysis is entered into the M&E plan and progresses against the outcome indicators are reported.

Overall, implementing a successful monitoring strategy should reduce down time and provide operational efficiencies.