Reflection Assignment #5 - Evaluation Reports

There are two main "takeaways" for me after reading the report.

The first is stated in the introductory section of this chapter.  It states "Designing, preparing, and presenting an evaluation report is the very essence of evaluation, but if the report does not prove useful or influential, then the 'essence' quickly evaporates into an empty exercise."

As I reflect on this thought it solidifies my preconception that the content and presentation of the evaluation results to the audience is critical to whether the evaluation has any value.  As the chapter points out, a careful and engaging report (both the printed version, and any oral presentations made) will bring about real affect of the evaluation in changes that may be required in the evaluation object.  It is also essential that these reports be useful and reliable for the profession to continue to justify the funds that are required to complete this work.

If evaluations are viewed as bureaucratic "hoops" rather than being useful to key stakeholders in improving their programs, then the value of effective evaluation will be lessened and opportunities for real improvement may be pushed aside.  Perhaps the greatest danger would be for evaluation to move from being an objective means of improving and strengthening programs to becoming an arrow in a quiver of those seeking to fight or support programs.  While this may create more opportunities for professional evaluators, it could taint the supposed value of evaluation, fair and independent assessment of the evaluation object.

Evaluation reports need to keep this issue in mind.  In addition to being engaging and comprehensible, it is important that they be fair and accurate.  As the chapter points out, the best evaluations will result in a fair description and assessment of both the strengths and weaknesses of the object being evaluated.  This means that here will be both positive and negative feelings about the evaluation from the different audiences to the report.  I have observed the point made in this chapter that mentioning of weaknesses will bring about defensiveness.  I have also observed that this is just a natural reaction and that part of the value of the evaluation team is to help the stakeholder work through this defensiveness to help them see the weakness and begin to make improvements.  This requires the right mix of people skills and strategic thinking on the part of the evaluator in order to "add value" to cement the effectiveness of the evaluation.

The second "takeaway" was not really covered in the chapter but was an inference I understood more clearly.   Once you express yourself in writing on paper, particularly in high-stakes evaluations, the words live forever in the context in which the ink exists on the paper.  It is impossible to prevent others from repeating their words and applying their own interpretations and context.  I was even more impressed by the need to be clear and concise in expression and to be economical in the choice of words.

While the chapter would remind us that it is important to be engaging in our prose and to use graphics and other methods of breaking up the monotony of the report, these can diffuse the clarity of the report and lead to vagueness that could be construed according to the desired objectives of the user or reporter of the information.  I also realized that this risk cannot be the overriding concern of the evaluator or the report will become too stale to have any benefit to the user.  There is a balance that must be accounted for so that the user engages, but that the evaluation report is clear in context and interpretation.

The last point that was helpful to me was the discussion of the influence, intention, and time frame postulated by Kirkhart.  The evaluation report will take on a life of its own once it is in the realm of publication.  The words will live forever and can be accessed at multiple points in the immediate, mid, and long terms.  The findings and influence of the report will live on not only in the implications for the immediate evaluation object, but also as it may be inferred to similar objects, and in the lives and experiences of the evaluators and the stakeholders involved with the particular evaluation.  The effects of change and influence from one evaluation will extend through the experiences and findings to other contexts, objects, and some elements of society through the learning that occurs and the influences on the stakeholders.  That is a valuable and astute observation.  In essence, each evaluation will change something, even if it is not the immediate object of the evaluation.