For this assignment we were asked to create a survey for our test program evaluation. Once we created the survey, we were to test it with four individuals and then re-work the survey based on the results we found from the testing.
The survey that I created for this assignment was for the students that are participants in the CAP program.
In creating the survey I needed to first reflect on what information I wanted the survey to give me. In doing this I decided to look back at the logic model that myself and the stake holders had developed for the program. My thinking in doing this was that I could see the ideal for the program. I wanted to see what the program was supposed to be doing, and then use the survey to help find out if it was doing those specific things.
The first area that I wanted to look at was why the students are in the program. I wanted to be able to determine if students were in the program because of a previous class that they had failed, if they are in the program for support in a class they are currently taking, or if they are in the program to work ahead.
Secondly I wanted to get the student’s opinion on how the CAP program environment differs from the environment they experienced in the traditional classrooms. I thought that this may give me some insight into why the Cap program is more or less successful to them than the traditional classroom is.
Thirdly, I wanted to determine if the CAP program was a success for these specific students. I wanted to know if their GPA had increased as a result of the program, or if they received more credits than they ultimately would have. And I wanted to see if the program had any effects on work habits. These are all outcomes that the stakes holders hope for, and that could be found in the logic model we created.
I have posted below this post both the original version of the survey as well as the completed version that was re-worked after the testing. The test subjects completed the first version of the survey and then I evaluated the results with the stake holders. In evaluating the responses from the first version of the survey, there were a few areas that we felt we should address.
The first area was that we felt we did not know enough about the student participants. We felt to really evaluate more accurately the effectiveness of the CAP program; we need to know more about the participants who entered the program. This prompted us to add the questions asking how many classes they were participating in CAP for, as well as asking for a GPA prior to entering CAP.
We also felt that we were not truly having the students rate the success of the CAP program because we were only asking about the GPA after the program. The GPA increase was not the only factor that impacts the success of the program; we also wanted to know if they received credits they would not otherwise have received. A question was added to determine if students felt they received more credits than they would have without CAP. We also added the opportunity for the students to list some skills the CAP program helped them develop.
The wording of a couple questions also gave the students some difficulty. They were asked in the original version about “support” they received from teachers, and they seemed to misunderstand the term “support”. We decided to change the wording to the “help / attention” they received from teachers. This seemed to convey are meaning better to the students.
Finally, the question about attendance was another area where we were not sure of the accuracy of the responses. We were not sure that the students were clear about the options for the frequency of attendance, so we changed them to be clearer.
This process was certainly a beneficial one, for two reasons. The first was just the overall process of creating a survey. Developing the questions gave me the opportunity to really examine the program and understand further what I wanted to learn about it. But beyond the experience of developing the questions, the experience of testing a survey to see if the expected results are returned was beneficial as well. It was good to see that what you expect to hear back is not necessarily what you will hear.
This is the draft version of the survey...
This is the Completed version of the survey...
Great work Mark. Glad to see you integrated a focus that was a part of your program plan. I can see by what you shared that you used the other material you have created to guide the initial and the redevelopment process.
ReplyDelete