Auditing and business strategy analysis share many concerns in common with evaluation in the context of this text. Among these similarities are the principle concerns regarding:
- The independence of the evaluator/auditor
- The role of internal vs external evaluators/auditors and their biases
- The role of the audit/evaluation as a report to inform, or a tool to influence
- The standards and rules governing the evaluation and the evaluation purpose
- The interplay between quantitative and qualitative measurements and their appropriateness for the evaluation need
- The objectivity or subjectivity of the observations
So what did I learn this week? Primarily I learned the classification of the different evaluation approaches.
- Objectives-Oriented Evaluation - the purpose is to identify the extent to which specific program objectives were obtained. This can be done by collecting data and analyzing the results to see if program goals are achieved. The most interesting part of this approach for me was the idea of goal free evaluation. I see many applications of this approach to identify what was actually accomplished without being anchored to the goals as they were outlined. I have encountered unintended consequences and unanticipated variances so often in my career that I see real potential in this approach to evaluating the accomplishments of the program.
- Management-Oriented Evaluation - this is the type of evaluation that I am most familiar with. Its purpose is to treat the decision maker as the primary stake holder and help them refine and identify the decisions that need to be made and then scope the data collection and analysis to supply the data needed to inform that decision. This approach reports the facts and its influence on the decision made is largely neutral. There is danger in trying to anticipate the decision or influence the decision. This bias may limit the factual credibility of the evaluation.
- Consumer-Oriented Evaluation - This approach uses checklists or other criteria in hopes of providing useful information to inform consumers. While a desire to remain objective is important, there are come who would argue that subjective concerns may benefit society as they lead consumers to more effective alternatives that will lead to change in society. The major growth in this area cam largely from the expansion in federal funding since the 1960s.
- Expertise-Oriented Evaluation - This approach uses an expert, a panel of experts, a formal, or an informal approach to evaluate programs. A most common evaluation event using this approach is accreditation of educational, medical, and other institutions. This is also the idea behind peer-reviewed refereed journals. The experts render an opinion on the effectiveness or validity of the object or program, particularly related to objectives, standards, or societal expectations.