Friday, 20 January 2012

ECUR 809 - Assignment #2

For this assignment we were asked to review a program outline provided to the class, decide which model of Program Evaluation we would use to evaluate the program, and explain why we chose the model we did.  The program outline that we were given was a Prenatal Exercise Program for Aboriginal Women in Saskatoon.
In reading over the outline of the program it seemed to be separated into three main sections; the description of the problems that were occurring before the program took place, the events that should happen during the program, and then the outcomes that this program hopes to experience.
Because I feel that this program outline was separated like this, I would use Stake’s Countenance Model to evaluate the program.  I would choose this model because it examines the congruence between what was intended by the program and what actually occurred in three areas; the “Antecedents”, the “Transactions”, and the “Outcomes”.  These three areas in Stake’s Countenance Model can be mapped to the three main sections that I found to be in the program outline.
In examining the Antecedents I would look at the items listed in the “Introduction” and “Background and Preparation” sections of the program outline.  These sections outline the conditions that are taking place among Aboriginal women before the program began.  I would look at the congruence between how this program outline document outlines the conditions prior to the program and how others may have explained the conditions prior to the program.
In examining the Transactions I could look at how well the observed program actions are congruent with what the program had planned to take place.  Did what the program outline said should be done in order to resolve these problems actually take place when the program was running?
And in examining the Outcomes I would look at if the observed program outcomes were congruent with what the program had intended to be for the outcomes after participants had taken part in the program.  
I think by using this model, in this case, the effectiveness of the program could be thoroughly evaluated in a meaningful way

Friday, 13 January 2012

ECUR 809 - Assignment #1

          For this assignment we were asked to pick a completed program evaluation, explain the model used, and determine the strengths and weaknesses.  The completed Program Evaluation which I will be examining is the Saskatchewan KidsFirst Program Evaluation which was completed in 2010 by the Saskatchewan Population Health and Evaluation Research Unit (SPHERU).  In completing this evaluation several reports were published, however the bulk of the findings can be found in the following three documents; The Report of the Qualitative Study http://www.kidskan.ca/node/279, the Report of the Quantitative Study http://www.kidskan.ca/node/280, and the Summary of Findings and Recommendations http://www.kidskan.ca/node/281. 

            The KidsFirst program is a program provided by the government of Saskatchewan to at risk families with young children in certain areas of the province.  The goal of the KidsFirst Program Evaluation was to, “assess the program’s effectiveness in helping participating families and communities make positive changes” (Summary of Findings and Recommendations).

            The Saskatchewan KidsFirst Program Evaluation does not employ one pure model of program evaluation.  However, you can clearly see the strands of Stufflebeam’s CIPP model being weaved throughout the evaluation.  The CIPP model looks into four main areas of the program when doing the evaluation.  These areas are the Context, the Input, the Process, and the Product. 

Context - The authors evaluate the context by clearly outlining the four main goals of the KidsFirst program, and the objectives underneath all four of those goals, and then each one of these objectives is evaluated.  This process is clearly evaluating the context of the KidsFirst program.

Input – The KidsFirst Program Evaluation also examines the way in which the strategies of the KidsFirst program are linked to the goals and outcomes that the program wants to reach.  It examines this mainly in the qualitative study by interviewing the program participants.  This helps the evaluation team examine how well the program strategies and the goals are linked.

Process - Evaluating the process is looking into the strategies, techniques, or daily activities that are done in order to reach the goals of the program.  In this specific program evaluation, the process is evaluated mainly by interviewing those who take part in the process.  The participants interviewed included the parents, those who conduct the home visits, the home visitor supervisors, and the program managers.  Each participant was asked questions regarding how the process currently works, and was asked if they had anything that they would change if they had the opportunity.

Product – The product of the program is evaluated by determining how well the program has reached its main goals and their sub objectives.  In order to do this it attempts to answer a variety of questions including, “Did families in KidsFirst develop and maintain a safe and secure home environment?” (Quantitative Study).  All of the questions that the evaluation asks, and ultimately answers, are tied to the objectives that the program has set out to accomplish.  By answering these questions, the Product of the program is being evaluated.

            The main strength of this evaluation, I believe, is its thoroughness.  In conducting this evaluation, the team published six different reports which focused on a specific area, and then finally submitted a report which was a summary of their findings and recommendations.  Dividing the report into sections with related material made it easy to read and find the required information which is definitely a strength.  And another point that I found to be positive was that the evaluation conducted both a quantitative and a qualitative study.  This fact allowed the team to make their arguments based not only on the data, but also on feedback from the participants of the program.

            I found the main weakness of the evaluation to be in its recommendations.  Maybe if I was someone they were directing them to, the meaning may have stood out a bit more.  However, I felt that some recommendations were fairly vague.  The recommendations were listed, but no real reason as to why this is being recommended, or how it may be implemented was discussed.

            I really enjoyed the opportunity to study this program evaluation.  It gave me the opportunity to take a first look at a completed program evaluation, and helped me to start to wrap my mind around the process involved in conducting such an evaluation.