How to evaluate a professor’s teaching

This is a very good question. I recently wrote a blog post on how professors are evaluated based on students reactions to the course (as in do they like you are not). Unfortunately when doing an evaluation, this is least important compared to other criteria (not that its not important just that in the grand scheme of things it doesn’t tell you much). In that post I discussed how you properly evaluate a course by doing the following:

Step 1 – Student reactions. Did students like the training, process, course, instructor? This is the lowest and least important level. It doesn’t tell us if the course, instructor, or training was effective at all. This is the only thing student evaluations measure. Thus faculty are critiqued on whether students liked them or not. Not whether they actually taught anything at all.

Step 2 – Learning. Did the students learn what was taught? This is pretty important. If they learned the content then the course was effective. This is not measured on student evaluations.

Step 3 – Transfer. Are students able to apply what was learned to their jobs? This is very important. This is not measured on student evaluations.

Step 4 – ROI (return on investment). Was the training worth it? What were the benefits of the training? This is extremely important. This is not measured on student evaluations.

Personally, I only look at step’s 3 and 4 when evaluating myself. They are difficult to obtain but I periodically check on my graduates and they tell me what was useful, what they got from my courses, and they thank me for courses or information that they may have not appreciated at the time but then did when they started their careers. This is the exact feedback we all need to hear. All of my courses and materials are geared towards a career in instructional technology and design – so they are all practical and project based. I teach the students how to work, handle/respond to situations and clients, and how to excel in their careers. So I hope that my students give me that type of feedback but you never know until you ask and unfortunately we are not asking. So when doing a professor evaluation we should be looking at those important factors, not just popularity. I should note that my reaction/popularity scores are always good. I just feel that they do not tell me anything.

Having said that, when I fill out my annual performance charts I usually only have step 1 from above, provided by the school along with some quotes from alumni who lead me to believe step 3 and 4 were met. However, I do not think those quotes are really examined as part of my teaching because they are qualitative and not everyone will take them seriously. So I write this to call on faculty to start demanding that we examine whether learning took place, if the students are using it, and was the class worth it. Otherwise why are we teaching it (the topic or course)? Maybe there is justification (as in its an intro course that is needed to get to the more advanced topics and thats fine). But it’s something we need to consider. The teacher with the highest step 1 may not teach the students anything and this is a problem. As a result I am personally going to start doing a few things…

1. Do a pre and post test of content in my courses. This will evaluate student learning and will be objective (true/false or mult choice) and will measure low and high level knowledge. This will measure course objectives. This will tell me if students learned or not as I can compare the scores.

2. Send out annual or biannual surveys to my alumni asking them for input into how my courses and program is impacting them. What information are they using? What didn’t they get from courses that they needed?

I believe this information should be required of all faculty but since it’s not I will start collecting it and providing it for myself for my peace of mind. Hopefully I can start a trend here into better accountability for my own teaching and that of others.

How I replaced Blackboard with 3D Gamelab

In today’s college classroom the LMS is commonly used to manage, track, and facilitate the learning process for both face to face and online learning courses. Most colleges currently run the Blackboard LMS, which is a very good LMS that is designed to meet the needs of today’s faculty. However, I personally have found Blackboard to be a bit dull because it’s used so often and I love to experiment with new things. So I learned about a new LMS called 3D Gamelab which was designed to function like a gamified LMS. Obviously given my interest in games and the fact that I teach a gaming course I was intrigued to find out more. So I got a copy and tested it for 3 semesters. I tested it in 5 courses during the Summer 14-Spring 15 semesters for my Instructional Technology graduate students.

What is it? 3D Gamelab is an LMS that is geared towards gamers. It functions just like blackboard and others but has some different features. For instance, badges, awards, and experience. There are no grades in 3D Gamelab. There is experience. So students rank up levels and earn experience as they complete assignments. This experience is then translated into your grade.

Assignments. Assignments are called quests. So each time a user logs into your course they see the quests that they need to complete. These can be big assignments – like here is your final project or small tasks like a quest asking users if they read the readings for the day. I found using it to have students confirm that they read the syllabus and such to be very valuable (something other LMSs do not really allow you to do).

Badges, awards, achievements. You can set up badges, awards, and achievements in the LMS. Badges are for skills that the students acquire, awards are awards for completing assignments and such, and achievements are for completing tasks. Technically all 3 can be used for the same thing if you wanted. I assign badges for skills such as being a beginner in HTML. These badges can then be transferred to the users Mozilla Backpack, which is awesome. I use the awards and achievements for things like completing an assignment, earning a rank, etc.

Grading. This is something that is both good and bad. It’s great because when I log in I can see what needs to be graded and students can see where they rank compared to the rest of the class. Also if students did not get a 100% on my assignment I return it to them and they will need to recomplete it for credit. The disadvantage is that you can’t give grades other than perfect. So if they got an 80% the only option is to send it back to the student to redo until they get a 100%. This is how games work which is why its set up like this however this makes it difficult for instructors. This works fine for my graduate classes which are project based and I rarely have students that do not get A’s. But this would not work for my undergraduate students who do not turn in perfect work, turn in things late, and do not always get A’s.

Price. It’s relatively inexpensive. The cost is around $100 a year per instructor for all of your classes. So trying it was a no brainer.

Usability. In my opinion is very simple to use and set up. However, I am a ‘techie’. I noticed that my older students tended to have trouble navigating and always seemed to ask me for help. This is not something I experienced with Blackboard but I did with 3D Gamelab. I think if the company offered a really good interactive tutorial that this problem would be solved. My younger graduate students had no issues.

Student reactions. My students loved it. They really liked it. There were a few glitches here and there but overall it was a good experience. They really like the badges and loved the experience/ranks/quests. The comments in their reflections were that it was different and was fun to use. However, they also noted that while they really liked this they thought that if it was used by the university for all of their classes they would probably lose interest quickly and just think it’s dull like they believe Blackboard is.

My recommendation. Try it out. It’s a lot of fun. It’s different. Your students will like it. It’s worth the money.

How will I use it in the future? I have decided that I will use it for two of my courses – gaming and project management. Both of these courses are set up as competition/gamified courses so it will work well. I am going to use blackboard for my other courses as I want to mix things up and I don’t want my students getting sick or tired of any technology in my classes. So variation is best.

Should you buy a smart watch?

With the Apple smartwatch just around the corner I thought I would offer some advice to those seeking one. I have owned a sony smartwatch 2 since 2013 and have had a chance to really see what these devices can do. Here are my thoughts…

Overall they are really cool gadgets. For a few hundred dollars (some are only $100-$150) they are not expensive and in a pretty good price range. You definitely do not need one though. I tell my students that they are the perfect gift for the ‘techy’ that has everything. Otherwise they probably are not worth it for most people unless you really want a new watch. The cool thing about them is that you do not need to take your phone out of your pocket, you can just check your wrist when you get a text. Essentially they are an extension of your cell phone. Another neat feature is that you can remotely control your phone. So if I am taking a family video/photo I can set up my phone and then use the watch at a distance to get all of us in the picture. That is the most useful feature I have found for the watch thus far and it is pretty cool although a timer on my phone works just as well. However a major concern with these devices is that they do release radiation (microwave radiation) so in addition to your phone you now have more radiation from another device. That is just a point to consider. I personally do not wear my watch too often for that reason and the fact that I do not like wearing watches. But I will admit they are kinda cool devices and you will not be upset with your purchase if you think you want one. But they are not something you need. They do not do anything your cell phone doesn’t and they require your cell phone to even work.

* I did want to add a quick note. There are many other kinds of watches out there for sports like surfing, running, etc. I am not discussing those in this post. The watches I am referring to are the android/apple based watches. The sports watches look awesome but I have yet to try them. A watch that calculates my waves sounds really cool as does a fitbit with GPS/heart rate data on it.

Using students evaluations to evaluate faculty

As my semester gets closer to the end I start receiving emails about end of the semester student evaluations. These evaluations are used to determine if faculty are doing their job (i.e., teaching). So should these be used to evaluate faculty? Well since part of my job in corporate settings was to evaluate training let’s compare what is done in corporate america to determine the success of the course vs what is done in education. In corporate we generally used Kirkpatrick’s evaluation model. As you are reading this keep in mind that student evaluations only measure step 1 of this model. So…

Step 1 – Student reactions. Did students like the training, process, course, instructor? This is the lowest and least important level. It doesn’t tell us if the course, instructor, or training was effective at all. This is the only thing student evaluations measure. Thus faculty are critiqued on whether students liked them or not. Not whether they actually taught anything at all.

Step 2 – Learning. Did the students learn what was taught? This is pretty important. If they learned the content then the course was effective. This is not measured on student evaluations.

Step 3 – Transfer. Are students able to apply what was learned to their jobs? This is very important. This is not measured on student evaluations.

Step 4 – ROI (return on investment). Was the training worth it? What were the benefits of the training? This is extremely important. This is not measured on student evaluations.

As you can see student evaluations are not very important. I mean they tell us whether students liked the course or not. However, thats it. They do not even tell us if students learned anything. Essentially students could rate a course where they learned nothing really high and vice versa making these kind of evaluations worthless when they are by themselves. Thus should faculty be evaluated with these? No. At least not as the only measure and certainly not as an important measure. It’s a pretty worthless measure. There is a reason that we use these other steps in corporate settings – we know that they are meaningful yet for some reason higher education has yet to recognize that.

Essentially what I am trying to say is that if I tried to rate a course or teacher in corporate settings the same way I did in higher ed I would be fired.

Do video games cause violence?

The short answer is no. Video games do not cause violence any more than sports, tv, or play (ie cops and robbers or tag). Here is a look at some of the research on the topic. Note that it tends to go both ways indicating that people are pulling numbers to support their views but not looking at the big picture:

-The 2008 study Grand Theft Childhood reported that 60% of middle school boys that played at least one Mature-rated game
-Violent juvenile crime in the United States has been declining as violent video game popularity has increased. The arrest rate for juvenile murders has fallen 71.9% between 1995 and 2008. The arrest rate for all juvenile violent crimes has declined 49.3%. In this same period, video game sales have more than quadrupled
-Increasing reports of bullying can be partially attributed to the popularity of violent video games.
-A 2004 US Secret Service review of previous school-based attacks found that one-eighth of attackers exhibited an interest in violent video games, less than the rate of interest attackers showed in violent movies, books, and violence in their own writings. The report did not find a relationship between playing violent video games and school shootings
-A 2000 FBI report includes playing violent video games in a list of behaviors associated with school shootings
-Playing violent video games provides a safe outlet for aggressive and angry feelings. A 2007 study reported that 45% of boys played video games because “it helps me get my anger out” and 62% played because it “helps me relax.“

Source of stats – http://videogames.procon.org/

But what about games like Grand Theft Auto where you can literally shoot people, hit them with weapons, or beat them up? Put it this way, Grand Theft Auto 5 has sold over 40 million copies. The biggest selling games in the US every year are Grand Theft Auto or Call of Duty. How many of those 40 million people went out and caused violence due to the game? Not only that but the latest GTA and COD games have sold many more copies than their originals in the early 2000s. In fact, the sales of these games are record breaking. Thus sales of these games has greatly increased yet guess what? Violent crimes have decreased. Am I just pulling numbers to support my conclusion? Yes however these are very telling. If these games were causing more violence, and the games are selling many millions of copies more now than 10 years ago, surely there would be noticeable spikes in violent crime.

Source of image – FBI.GOV – http://www.fbi.gov/news/stories/2014/november/crime-statistics-for-2013-released/crime-statistics-for-2013-released

Now having said all of that, I think that video game ratings are important and that kids should not be playing Call of Duty or Grand Theft Auto. They are too young to always know what is right/wrong and appropriate or not. So parents, games have ratings, please use them. If a child is very mature then I would have no problem giving them these kind of games but I do not think that most kids are ready for them. I think its a case by case basis and up to the parents. keep in mind though that kids are playing these games even though you have to be 18 to buy them so I encourage parents to really watch and monitor what their kid plays. I know that I will be.

 

 

Computer Adware, Viruses, Malware, and Security

Have problems with any of these? Here is a list of steps and freeware you should use to fix and protect your machine. Please note this is for PC only and all software is free:

1. Run rKill – This will stop all of the bad stuff from running so that you can run other software to delete it.

2. Run Malwarebytes and Ad-Aware

3. Run Combofix

4. Run Antivirus and keep it installed/running – ClamAv

5. Install security and keep it installed/running – Zonealarm

Once you have scanned your computer with the first 4 it should be clear. I would then recommend step 5 and keep it running on default settings. Also keep ClamAV running. The others should be run and used as there are problems on your machine.

 

CHMOD Commands – Permissions an LMS administrator should know

You must understand permissions if you are managing an LMS (as some of my students will do when they graduate). Thus I am posting this from last night’s class:

›Three types of users: User – Group – World

These permissions can be given for one group, multiple groups, one user, etc.

Three types of permissions: ›Read (R or 4), Write ( W or 2), Execute (X or 1)

Sometimes written as – RWX or a number equivalent such as 777. Each word corresponds to a number depicted below that can add up to 7 (4+2+1).

›R- Read (4) – read files

›W- Write (2) – write/modify files

X – ›execute (1) -Read/write/delete/modify/directory

›Example –

›-r–rwx—- = User can read, group can do everything, world cannot access

›777 = Everyone can do everything

›422 =User can read, group can write, world can write

›Rwx-rwx-rwx = Everyone can do everything

›-wx-r—– = User can write and execute, group can read, world cannot access

ATT UVerse Review – Avoid this company

Since I signed up for ATT UVerse and switched from Comcast I have noticed some interesting things that others should be aware of before they get this service. So here are my thoughts. And before you read this realize I am not one to complain about this stuff but I have never had such bad customer service in my life:

1. Do not believe anything they tell you. In fact, they will actually lie to you – well they did to me and I still just cannot believe how I was treated by this company. When I signed up for the service I was promised 1 year of HBO/Cinemax for free – nothing I care about but it was promised to me. I asked the person on the phone several times if this was for the year and she said yes. Keep in mind I actually said ‘this is for the full 12 months of my contract’ and she said yes. I asked this question like 2-3 times just to make sure that I understood correctly and she wasn’t giving me one of those 3-6 month deals that would end and increase my bill. Three months later I realize my bill jumped $50. So I call ATT and they say there is nothing they can do. I ask them to listen to my previous phone call (as they record them) and they said they would and get back to me within 5 business days. I thought great, they dont believe me but once they listen to the call I will get what they promised. 7 business days with no phone call from ATT I call them up and they apologize and tell me a manager would listen to the call asap and get back to me. 7 business days after this with no call I call again. I am told that no one is going to contact me and that even if this person promised me a free year of HBO/Cinemax that the company was NOT going to give it to me. Wow. ATT will be dropped very soon for this. Customer service gets an F.

2. Internet – I pay for up to 18mbps. I average around 6. This is terrible. No way on off peak hours should I be getting such terrible internet.

My recommendation – Avoid uverse like the plague. Comcast isnt much better but they never lied to me and my internet was always around 13mbps when I paid for 15. So what have I done? At this point I had to drop a bunch of services that were promised to me in order to keep my bill at the original cost. A real bummer. I had high hopes when I heard of Uverse but its a real let down. Oh well at least people now know not to sign up with them. I think the best thing to do is to cut the cord altogether.