There is so much to learn from many different theories and practices of program evaluations from many evaluators. Evaluation is, “Both science and art…..no one methodology exists. Each situation presents its own particular needs”.
Program evaluation is a process that involves a systematical collection of information from various activities, the outcomes and characteristics of a program in order to ascertain the ways through which it can be enhanced and made more effective; or more often it is defeined as a systemic collection of information about the activities characteristics and outcomes of programs to make judgments about the program, improve program effectiveness and/or inform decisions about future programming”.
The evaluation process also defines it as the information that may be used hypothetically to make the decisions in relation to future programs. Evaluation methods are designed to serve three primary purposes, planning, development and accountability of the program. There are various approaches/models that can be utilized to evaluate programs, depending on the type of the program the purpose, and the needs for the evaluation process. The aim of this essay is to outline basic approaches and to analyze and compare various models of program evaluation; summative and formative evaluation, developmental evaluation, utility focused evaluation, participatory evaluation and theory driven evaluation.
The method, of the evaluations influences the was billions of dollars are spent.
The ultimate goal for the utilization of the money is not taken into consideration. These programs are social in their nature and are often utilized to curb poverty, crime, social inequality and health-related problems of the humanity. The human based social programs that combat various illnesses in the society need to be assessed in order to distinguish the effective ones and those being not effective to enhance the efficacy of the program. This is a main reason why evaluation programs are launched. Program evaluation is closely related to policy analysis, as policies are implemented through various programs activities. Program evaluator has to analyze/ evaluate the program to understand why a particular policy was more or less successful. There is often a disconnection between the policy making process and the outcomes, or in other words the ultimate results of the policy. Evaluating social programs has also implications for designing policies that would reflect the needs of program users/program participants and the stakeholders. “Evaluation is by its nature a political activity. It serves the needs of the decision makers, results in relocation of resources, and legitimizes who gets what. It is intimately implicated in the redistribution of basic goods in society. It is more than a statement of ideas; it is a social mechanism for distribution, one which aspirers to institutional status. Evaluation should not only be true; it should also be just. Current evaluation schemes, independently of their genuine value, reflect justice differently. And justice provides important standards by which evaluation should be judged.
Program evaluation is similar to the engineering process, i.e. connecting the theory based evaluation methodology to practical issues and implementation . Unfortunately, many reports written after these evaluation processes normally end up gathering dust on bookshelves. Many of them remain unread and never used. This essay examines various approaches of the evaluation in order to establish the one that is considered to be the most useful and mostly utilized by the international community. Besides, the similarities between various approaches for evaluation process are subjected to meticulous analysis
Evaluation Issues
For the purpose of evaluation it is important to distinguish between social research and evaluation. Both, Hughson (2003) and Patton (1997) has pointed out that, while evaluation is closely related to the traditional social research, two processes shall be distinguished and have different purposes. It has been stressed that the purpose of a research is to discover new knowledge, establish truth, test theories and generalize across time and space. The aim of a program evaluation is to communicate resolutions , ascertain alternatives , outline enhancements , and to provide information about programs and policies within the frontiers and limits. Both, evaluation and the traditional social research use methodologies that are similar outwardly . However, due to the fact that evaluation occurs only within an organizational and political context, political dexterity, the ability to manage a program, group skills and being sensitive to numerous stakeholders are required for a success. Generally, the social research does not require many of these skills. In this essay, I will look at a general idea of what evaluation is being contained of, as well as the issues involved in the field and the terms used.
The ultimate goal of many evaluation processes is to come up with a feedback which can be useful to stakeholders. These stakeholders include but are not limited to administrators, sponsors, donors, client-groups, staff as well as other constituencies being relevant to the process. This feedback is quite useful to these stakeholders because they generally rely on the feedback to take important decisions. “Most evaluators…describe values held by stakeholders, determine criteria they use in judging program worth, find out if stakeholders think the program is good, and see what they think should be done to improve it. The claim is not that these values are best; but that they are perceptions of program worth that grist against for the mill of decision making…Stakeholders based approaches…solicit information about stakeholder interest in the program and its evaluation”.
Researchers generally have a convergent opinion that a link between the process of evaluation and the impact it has on the program is often complicated. This means that the feedback does not always influence the decisions made in a short term. Even though it does not always influence on such short term decisions, the main goal of conducting the evaluation is to help making both short- term and long-term decisions.
Formative and Summative Evaluations
There are primarily two types of evaluation that could be done for different purposes. Scriven identifies two alternatives; formative and summative evaluation. Formative evaluations improve program or performance by influencing immediate decisions about the program, especially about how its component parts and processes could be improved. Summative evaluations judge program worth by assessing program effects in light of the relevant problems. Formative evaluation is a kind of evaluation that facilitates to improve a policy or a program. Summative evaluation, on the other hand, is a type of evaluation that is useful in assessing the value of an evaluation program. As Patton (2002) suggested, it is important that the merit or the worth of the program has been assessed in order to determine whether or not the program should be sustained, scaled up or discontinued. However, the two types of evaluation, summative and formative evaluations, are very of considerably rigor . The two methods are only effective when used on the work being definite by nature. Definite work can be paused in order to allow for the evaluation to be conducted.
Campbell (2002) confirmed that there are numerous types of evaluations that can be used. The choice of the type of evaluation to be used depends on the object that needs to be evaluated. Also, it depends on the reasons why it needs to be conducted. According to Campbell (2002), there are two main types of evaluation, i.e. summative and formative evaluation. Formative evaluations are very useful because they are able to improve or strengthen the object evaluated. Through formative evaluation, the way in which the program is delivered, is being examined as well as the technology used and the quality of implementation are being assessed.
Formative evaluation also examines the context of any organization, the kind of the personnel employed by the program, the procedures used as well as inputs. Summative evaluations, on the other hand, examine the outcomes from the process. Formative evaluation is focused on the summary of everything that happens after the program or some kind of technology is delivered. The assessment is being conducted to determine whether the program has achieved the desired results or not. The overall impact that the program has had on people or a community is also assessed and the estimated costs of the entire project are being evaluated.
According to Patton (2002), formative evaluations are typically conducted in the course of the program, even as the implementation process is ongoing. The main reason for conducting formative evaluations is to improve the program or to correct some problematic areas. The program is being examined and described in details. The outcomes of the program are also considered. Summative evaluations are different from formative ones because, unlike formative evaluations, they are only conducted on the programs being well-established. They allow policy makers to arrive to important conclusions concerning the program and decisions concerning the future of these programs.
Developmental Evaluation
Patton (2010) describes developmental evaluation as evaluative processes, including asking evaluative questions and applying evaluation logic to support program, product, staff and/or organizational development. The evaluator is part of a team whose members collaborate to conceptualize, design and test new approaches in long term, ongoing process of continues improvement, adaptation and intentional change. The evaluator’s primary function in a team is to elucidate team discussions with evaluative questions, date and logic, and facilitate data based decisions-making in the developmental process. Developmental evaluation is an approach to evaluate the focuses on the outcomes it provides. It has been found to be useful especially in the situations where the outcomes are being expected to change from time to time. Developmental evaluation has emerged as being particularly suitable for the programs being in the process of development. Developmental evaluation is also suitable when the innovators of a project wish to establish the connections existing between social changes that are short-term and those that are long term ones. When innovators are working on the project that is continuously evolving, they do not have a model standardized and the developmental evaluation is the best approach for them to use. The right purpose and goal of developmental evaluation “should be to get social innovators who are, often by definition, ahead of the evidence and in front of the science, to use tools like developmental evaluation to have ongoing impact and disseminate what they are learning.
Developmental evaluation mainly focuses on the documentation of the decisions made and formalizing the lessons learnt. It is the knowledge gathered from these studies that will determine the decisions made by innovators. Developmental situations normally require one to keep adapting to changes, to look beyond slow improvements and to respond to the changes in the environment. The people involved in these kinds of projects are normally the innovators that are willing to test new things. Developmental evaluation is designed for such people that desire to go where no one else has gone and tried the things that no one else has tried doing in the past.
It was not until recently that the concept of developmental evaluation finally has taken off. The success of developmental evaluation actually coincided with the growth of systems thinking and the complexity of the scientific approaches. These are now being applied to the examination of social services and health-related services as well.
Developmental evaluation serves the needs of innovative programs that are not like programs or projects which are standard. People that innovate are not like those people implementing regular programs. Most of the time, innovators need some things to move very fast. Such value feedbacks are rapid and quick. They are a group of people that tolerate ambiguity. They are quick to embrace uncertainty and willing to adapt to new conditions, which keep changing and resist targets that are present. The outcome of innovation is never clear or measurable from the beginning. Innovators are motivated by vision and no other factors are considered by them.
Developmental evaluation is an approach to evaluation that is developed specifically for innovators. This is because innovators require an approach to evaluation that fits into their innovative spirit and their fast way of doing things. These kinds of people feel comfortable tackling dynamic. The program systems, which are complex and dynamic, are the kinds that characterize the world in which these innovators are working and living. They work within the developmental evaluation approach that can deal with the complex nature of their job. This is true because many of evaluation practices that are based on traditional evaluation approaches cannot manage the challenges that come with a complex process. Complex projects/programs are the ones where it is not very clear how the desired results will be achieved based on the high level of uncertainty of the outcomes. This is especially important when evaluating programs that involve human factors and programs based on the relationships. The expertise of the evaluator in such evaluations might help but the relationships are the key factor in evaluation process. In such projects/programs, the main stakeholders disagree from time to time about the things that need to be done and how they should be done. Such projects cannot really be controlled as they operate in the environment where many dynamic factors are constantly interacting. The projects are nonlinear which means that actions considered as small are able to produce reactions that are very large. There is also the emergence which involves the occurrence of patterns as a result of interacting agents. There are also adaptations which are quite dynamic. They involve agents and elements responding and adapting to each other.
It is the developmental evaluation, in Worthen’s view, which is able to meet the necessities of the innovators. This is because developmental evaluation allows innovators to apply complex concepts able to enhance the use of innovation. Developmental evaluation is the approach that allows social innovators to focus on an item or a project developed and to engage more into the process of innovation. Developmental evaluation is the approach to the evaluation which allows for a program to be evaluated during the real time and dynamic process. This means that when developmental evaluation is being used, the program is evaluated as it evolves. Developmental Evaluation is also able to adapt to the complex systems of these innovations. Developmental Evaluation monitors and assesses:
- The changes in the relationships between the components of a system.
- The appropriateness of the boundaries we use for the components of the system.
- The different perspectives about what changes and how it changes through a
developmental evaluation.