This is a very good question. I recently wrote a blog post on how professors are evaluated based on students reactions to the course (as in do they like you are not). Unfortunately when doing an evaluation, this is least important compared to other criteria (not that its not important just that in the grand scheme of things it doesn’t tell you much). In that post I discussed how you properly evaluate a course by doing the following:

Step 1 – Student reactions. Did students like the training, process, course, instructor? This is the lowest and least important level. It doesn’t tell us if the course, instructor, or training was effective at all. This is the only thing student evaluations measure. Thus faculty are critiqued on whether students liked them or not. Not whether they actually taught anything at all.

Step 2 – Learning. Did the students learn what was taught? This is pretty important. If they learned the content then the course was effective. This is not measured on student evaluations.

Step 3 – Transfer. Are students able to apply what was learned to their jobs? This is very important. This is not measured on student evaluations.

Step 4 – ROI (return on investment). Was the training worth it? What were the benefits of the training? This is extremely important. This is not measured on student evaluations.

Personally, I only look at step’s 3 and 4 when evaluating myself. They are difficult to obtain but I periodically check on my graduates and they tell me what was useful, what they got from my courses, and they thank me for courses or information that they may have not appreciated at the time but then did when they started their careers. This is the exact feedback we all need to hear. All of my courses and materials are geared towards a career in instructional technology and design – so they are all practical and project based. I teach the students how to work, handle/respond to situations and clients, and how to excel in their careers. So I hope that my students give me that type of feedback but you never know until you ask and unfortunately we are not asking. So when doing a professor evaluation we should be looking at those important factors, not just popularity. I should note that my reaction/popularity scores are always good. I just feel that they do not tell me anything.

Having said that, when I fill out my annual performance charts I usually only have step 1 from above, provided by the school along with some quotes from alumni who lead me to believe step 3 and 4 were met. However, I do not think those quotes are really examined as part of my teaching because they are qualitative and not everyone will take them seriously. So I write this to call on faculty to start demanding that we examine whether learning took place, if the students are using it, and was the class worth it. Otherwise why are we teaching it (the topic or course)? Maybe there is justification (as in its an intro course that is needed to get to the more advanced topics and thats fine). But it’s something we need to consider. The teacher with the highest step 1 may not teach the students anything and this is a problem. As a result I am personally going to start doing a few things…

1. Do a pre and post test of content in my courses. This will evaluate student learning and will be objective (true/false or mult choice) and will measure low and high level knowledge. This will measure course objectives. This will tell me if students learned or not as I can compare the scores.

2. Send out annual or biannual surveys to my alumni asking them for input into how my courses and program is impacting them. What information are they using? What didn’t they get from courses that they needed?

I believe this information should be required of all faculty but since it’s not I will start collecting it and providing it for myself for my peace of mind. Hopefully I can start a trend here into better accountability for my own teaching and that of others.