Evaluation

Evaluation attempts to objectively measure whether we have achieved our original programme objectives. We would use this phase to measure whether the behavior we sought to influence in our planning phase has been achieved, and what contribution our communication efforts contributed to achieving these results. Ideally this measurement is made against original baselines collected at the outset.

• Outcome evaluation focuses on whether strategic communication results, usually stated in terms of behavioural or social changes, have been achieved within a given time period; and

• Impact evaluation assesses the sustainability of changes identified in outcome evaluations some time after a programme has ended, determines whether its overall goal has been achieved, and analyzes the contribution strategic communication has made to this achievement. 

Some evaluation indicators will be similar to our monitoring indicators. An Evaluation will determine whether our interventions were sufficient and relevant to achieving short, medium and long-term results.

Steps in carrying out an evaluation:

Establish evaluation indicators

The following could form evaluation indicators on an EPI/NIDs programme:

Knowledge

• Percentage of target audiences reached by messages.
• Proportion of target audiences who know the national immunization schedule.
• Percentage of target audiences who can give two correct reasons why NIDs are conducted.
• Percentage of target audiences who know the linkage between routine immunization and NIDs.
• Percentage of target audiences who can correctly define AFP.
• Percentage of target audiences who can correctly state why community members should report cases of AFP.
• Percentage of target audiences who can correctly describe the process of reporting AFP.

Attitude

• Percentage of target audiences who believe that children should be immunized.
• Percentage of target audiences who believe that the immunization given during routine immunization and that given during NIDs is:
• The same
• Both safe
• Both necessary

Behaviour

• Percentage of children under five receiving OPV3 and NIDs immunizations.
• Percentage of immunizers giving caretakers key immunization messages at immunization points.
• Partners and organizations participating actively in NID activities, e.g. by attending at least 80 per cent of committee meetings in between NIDs or volunteering time to carry out EPI duties.
• Percentage of people reporting AFP cases.
• Number of AFP cases reported.

Sustainability

• Level of political and leadership support.
• Level of funding by government and other sources.
• Structures established to support communication at the different levels: national, district and community.
• Level of institutionalization of research-based communication processes.
• Quality of long-range communication programmes developed.
• Level of community involvement in planning, funding, implementation, monitoring and evaluation of communication activities.
• Level of decentralization of resource allocation and programme management.

Establish evaluation objectives

Based on evaluation indicators, and other concerns for the project, develop evaluation objectives. The following questions should help in the development of evaluation objectives:

• How do we want to use the findings of the evaluation?
• What decisions do we want the information from the evaluation to help us make?
• What must we measure to get the information we need?
• How much funding do we have for the evaluation? (Tailor your evaluation size and com- plexity to the funding available.)

Determine design and methods

Choose a combination of research methods and sample size in line with the information sought in the objectives.

Collect data

The data collection format used will depend on the research design.

Analyse data

Analyse data to indicate changes in knowledge, attitude and practices.

Utilize research data

Generally, research and evaluation data to guide focused planning is lacking in EPI Communication programmes. It is also true that sometimes research data is available, but is not used. A common reason for this failure is that programme planners feel overwhelmed and would rather wait for another day "when there is time to read and systemize research data."

To facilitate use of data collected during research and evaluation:

• Develop a summarized version of key findings and their implications for planning.
• Disseminate the findings to EPI team members and collaborating agencies. A good forum for dissemination may be the ICC meeting or a special dissemination meeting.
• Use the findings to develop improved strategies.
• Guided by research, monitoring and evaluation findings, move on to the next phase of programme development.