Category Archives: Higher Ed

Learning as a dependent variable in experimental research

Posted on by

I am reading through a number of peer reviewed studies and noticing that there are pieces missing from each. So if you are conducting an experimental study where learning is a dependent variable you must do the the following (note these are the things I keep seeing that are missing):

  1. Pre test. This is to ensure that differences found in your study are due to the condition and not prior knowledge. Be aware that if you give the same pre test as post test, its a violation of validity but it can still be done (its better than no pre test). Thus I recommend giving a different pre test.
  2. Content Validity. Is the content you are using correct? And is the assessment measuring the objectives from the content? How do you know this? Explain this. Has an expert reviewed it? Have you piloted it?
  3. What is the reliability of the test? Please provide us this information.
  4. Provide the M, SD, and Effect.

I know all of this seems like common sense but I am seeing top tier journals publishing articles that do not contain this basic information.

Are boring discussions taking over your online course?

Posted on by

If the online courses for CIA exam prep involve online discussion (and they should!), I urge you to consider the types of discussions your students are having. Before I get into that, lets discuss the advantages of online discussion. Online discussion is what separates online courses from traditional distance learning correspondence courses. If you are interested in taking online business related courses, check out this igcse online tuition in Malaysia. It gives us that ‘feel’ i.e., social presence, that a face to face course has. So if you want your learners to be happy there needs to be an element of discussion and student/teacher presence in the course. However, what I find is that many online instructors have boring discussions causing a slight decrease in satisfaction when comparing their online vs face to face student satisfaction scores. In fact at the end of my online courses I get many comments from students stating that they were happy they didn’t have boring discussions like other online courses they have taken. Many say they were nervous to take my course due to bad experiences in other online courses.

Why does this happen?

We know the students need to do the discussions as part of the learning process however the students usually see most discussions as busy work – something they are required to do, that is not fun, that they feel they do not get a lot out of. As a result, I have been experimenting with a discussion strategy that I believe works very well in online courses, which promotes learning and makes each discussion very beneficial for the student.

What do I do?

Instead of posing a discussion question that I hope sparks students interest I have students do a project. While we all love discussion questions the issue is that in a 5, 8 or 15 week course where students need to participate each week, there is no way you are going to come up with 1-3 discussion questions each week that students generally care about and thus they lose motivation doing the same thing over and over again. In addition, many discussion questions don’t even spark discussion and students are posting simply to meet the requirements in the course. This is boring! So I create a project for each week of the course that students post to the discussion forum and then discuss. I love this idea because students will get to see how another student viewed the assignment and learn from what they did. What are some example projects/activities that I have students do instead of a weekly discussion question? Some examples include presentations, videos, screencasts, critique articles/videos, charts, collages, short fictional stories, and many others. I try to make each week a different activity that relates to the content covered for the week. This way before my week starts students have done a small project on the topic and they have started discussing it. What I find is that many students go in and look at each students project because they are curious to see what their classmates did. This creates a great learning environment and makes students very comfortable with each other. I find that doing this has increased my end of course review scores, end of course comments, and increased the amount of compliments I get for my online courses. Give it a try and see how it works out for you. And I find this works for both undergraduate and graduate courses!

Back to school computer

Posted on by

What computer should you buy your son to be college freshmen?

First, How much money can you spend? Unfortunately when buying computer you usually do get what you pay for. So if you are spending $400 on a computer expect it to deteriorate faster than a $2000 machine (as in it might last a year vs the $2000 machine that will last for 4 easily). Also, expect that the $400 computer components are probably already 3-4 years old and will have trouble running current software. A minimum I usually tell people is to spend $700 and look at sites like https://www.hotrate.com to find the reviews of the best systems and compare their reviews to choose the best. Less than that and you are asking for trouble. $1000-$1200 is even better and what the majority of students need. And $1200-2000+ is what you need for a high end machine for gaming, computer programmer, graphic artist, movie editor, sound editor. Once you have decided your price point, then decide… Also if you are coming back to school. There are sites online that can absolutely help you throughout the year. You can finally look up advanced chemistry or advanced algebra with ease, or even the specific things such as the difference between then vs than for English.

Do you need a Mac or PC? Well maybe Macs are already out of your price range. So you can skip to the next paragraph. If you are not sure which you should get ask yourself the following: Are you good with computers (tech geek) or going into computer science, engineering, business or a science field? Then I would get a PC and if you come across any issues with your PC, this highly rated repair service can help you out. So much software only runs on a PC so you dont want to get into a class where something you are using wont run on your computer. If you are not good with computers and just need something that will last for your 4 years to write papers, browse the web, and be used in courses, then definitely get a Mac. They are great for those that are not tech people and usually will last 4+ years. I had one that lasted 8 years.

Next you need to decide what brand, upgrades, and where to buy…First, shop around. Look online and at stores. Take a sales associates words with a grain of salt. First they are trying to make a sale and they are NOT usually computer experts. If they were computer experts they wouldn’t be a sales associate at Best Buy or Walmart they would be working at a company making 60-90k a year. Secondly, they are NOT in your program and do not know what you really need. So do your own research. There are tons of back to school sales and all of the computer companies offer discounts for students. So look and compare. But what brand? If you are getting a Mac then you know you are getting a Mac. If PC however, there are tons of options out there. I would personally rather look components and price rather than brand per se but I do like Dell and Alienware (a dell brand) but there are many others out there too.

So what components should you be looking at? Disk space. How much do you currently use? I would recommend at least 250-500GB. Also remember there are free options for space such as dropbox so space might not be too much of an issue anymore unless you have tons of pictures, songs, movies (which all college kids have). Remember that if you run out of space you can buy a 1TB (1oooGB) hard drive for under a $100 at any time so don’t worry too much about the space issue. Next you have Ram. Get a minimum of 8GB but I would probably not get under 12GB if I wanted this to last for 4 yrs. My current computer has 32GB (and its 3 yrs old) just to put what you are buying into comparison of what is out there. Ram is one area where you need to upgrade. If you are a gamer or in a major where you need a fast computer get as much ram as you can afford. Then of course you need to consider the video card and processor. I usually try to get the best I can here as well. The least important thing for me is disk space as I can always upgrade that and buy more. As far as screen size, it doesn’t really matter – that is your preference. Get what you are comfortable with.

Should you buy a warranty? Yes. Please get it. I watch computers fail all the time. Please get it for at least 3 yrs if possible. Computers, especially laptops will wear out as they are carried from class to class and stuffed in a backpack. Keep in mind your school will fix computers as well so make sure you check with your school to see what they offer.

Should you buy Office and other software as an add on? NOOOOO. Wait until you get to college. Colleges will offer huge discounts on this stuff. For example you can buy the full office package at my university for $30. They have tons of software too so make sure you check before putting out big bucks on software.

Oh, and if you are wondering the answer is NO, a tablet cannot replace a computer. A tablet is a great compliment to a computer but you need a computer. You cannot get by with just a tablet.

How to get into college

Posted on by

So you want to go to college? This post is not for the 18 yr old high school senior, rather, it’s for the person that didn’t go to college right away (or failed out at some point) and wants to go later in life. Later in life can be 20 yrs old or 60 yrs old. But this post will still be helpful to those college seniors as well it’s just not geared towards them. Why am I writing this? I teach in a graduate program and have a wide variety of students applying and I wanted to share my advice for those going back for a bachelors that they never earned or for their Master’s. So…

*here is my post on should you go to college which you may want to read first

1. Relax. I understand you are nervous but just relax. It’s school. If you want to go, then go. There is a program out there that will suit your needs.

2. It’s never too late. Again – It’s never too late. You can change careers or paths at any age. Will it be easy? Maybe not but I bet you any amount of money it’s possible. So yes it’s never too late.

3. Why? Why are you going? Do not just say you want to go to college and have a career. What is it you want to do? What do you enjoy? What are your hobbies? If you were independently wealthy and could choose any career what would it be? Then select a field/major based on that desire.

4. Money. I don’t mean cost of college. What kind of job will you get with your degree? Now I don’t think you should get a degree just for money, college is about learning and learning to problem solve. But you need to have some career path in mind when you choose your major. You also need to figure out what kind of corporate uniforms you see yourself in in the future. How much will you make? Once you know what you will make then you can start choosing a college…So next is

5. Cost. How much will it cost? The average undergraduate degree in the US costs 30-40k total. So its about as much as an SUV and spread out over 4 years. If you are looking at a public university you should expect to pay that much in tuition. Plus you need to consider your living expenses and health care and special products like clear nail plus for nail fungus infection. Its very easy to work part or full time and go to college. However, the more you work, the longer it will take to get that degree. And certain degrees, like biology will be more demanding and offer less time for work. So you will need to figure this out. Remember that there are tons and tons of scholarships out there, like the ucla undergraduate tuition and fees scholarship fundings. Also, there are tons of jobs on campus that you can have while you go to school which may even pay your tuition. If money is your barrier than you are not doing enough research and planning well enough. If you are looking at schools that cost 50k a year in tuition and money is a problem for you then you are already doing it wrong.

6. Online or face to face. Do you want to go online or face to face? Do you live near a university? There is tons of research to be done here. I always recommend going face to face if you can because of access to campus facilities and professors. Even if you go online at your local university that would be better than going online at a university across the country. Access to the campus is key plus you can always mix up your online or face to face classes, which is the best of both worlds. Also, keep in mind if you do decide to go online that its harder for many people. Why? You need to have a lot of self motivation to be successful online. It takes a lot of self discipline. So consider that. There are no right answers here. Just what works best for you.

7. Finding a school. What is the schools reputation? Who are the professors? Do the professors have websites? Can you see their resumes? If you cannot find this stuff out from the website I would be very hesitant to apply. You should be able to see everything online without needing to talk to anyone. Look at my post on finding a college here

8. Will I get in? Yes you might need to take some tests. Yes you will need to gather information. Yes you will be nervous. My advice, your past matters but only up to a certain extent. Professors want students in their programs that are motivated to do really well and will put in the effort to do well. If you have a rocky past (bad grades, bad test scores, prior arrests, etc.) do not assume you cannot go. You can. You just need to make sure you are showing them that you can succeed. How can you do this? Set up a meeting with the professors and talk to them. This helps a lot. If they have talked to you and know you are really trying and motivated they might let those other things slide. Also, if you do get rejected and you really want to go, sign up for a class. You can always take classes without being admitted. In that class do really well then reapply. I bet you will have no problem getting in after that. Now all of this is case by case and some programs have tough requirements but if you want in, you will find a way. Watch the movie Rudy if you don’t believe me.

9. Tests – yes you might need to take them. My advice is to to prepare, buy the book, take a few weeks and really study. If you do bad don’t worry just follow my advice is step 8. Some people are just prone to doing bad on these. There are ways to study and I will write another post on how to do that.

10. Have a plan. How long will it take you to graduate? Know this. Know exactly when you will be done. Then you can plan things like internships and such.

How to evaluate a professor’s teaching

Posted on by

This is a very good question. I recently wrote a blog post on how professors are evaluated based on students reactions to the course (as in do they like you are not). Unfortunately when doing an evaluation, this is least important compared to other criteria (not that its not important just that in the grand scheme of things it doesn’t tell you much). In that post I discussed how you properly evaluate a course by doing the following:

Step 1 – Student reactions. Did students like the training, process, course, instructor? This is the lowest and least important level. It doesn’t tell us if the course, instructor, or training was effective at all. This is the only thing student evaluations measure. Thus faculty are critiqued on whether students liked them or not. Not whether they actually taught anything at all.

Step 2 – Learning. Did the students learn what was taught? This is pretty important. If they learned the content then the course was effective. This is not measured on student evaluations.

Step 3 – Transfer. Are students able to apply what was learned to their jobs? This is very important. This is not measured on student evaluations.

Step 4 – ROI (return on investment). Was the training worth it? What were the benefits of the training? This is extremely important. This is not measured on student evaluations.

Personally, I only look at step’s 3 and 4 when evaluating myself. They are difficult to obtain but I periodically check on my graduates and they tell me what was useful, what they got from my courses, and they thank me for courses or information that they may have not appreciated at the time but then did when they started their careers. This is the exact feedback we all need to hear. All of my courses and materials are geared towards a career in instructional technology and design – so they are all practical and project based. I teach the students how to work, handle/respond to situations and clients, and how to excel in their careers. So I hope that my students give me that type of feedback but you never know until you ask and unfortunately we are not asking. So when doing a professor evaluation we should be looking at those important factors, not just popularity. I should note that my reaction/popularity scores are always good. I just feel that they do not tell me anything.

Having said that, when I fill out my annual performance charts I usually only have step 1 from above, provided by the school along with some quotes from alumni who lead me to believe step 3 and 4 were met. However, I do not think those quotes are really examined as part of my teaching because they are qualitative and not everyone will take them seriously. So I write this to call on faculty to start demanding that we examine whether learning took place, if the students are using it, and was the class worth it. Otherwise why are we teaching it (the topic or course)? Maybe there is justification (as in its an intro course that is needed to get to the more advanced topics and thats fine). But it’s something we need to consider. The teacher with the highest step 1 may not teach the students anything and this is a problem. As a result I am personally going to start doing a few things…

1. Do a pre and post test of content in my courses. This will evaluate student learning and will be objective (true/false or mult choice) and will measure low and high level knowledge. This will measure course objectives. This will tell me if students learned or not as I can compare the scores.

2. Send out annual or biannual surveys to my alumni asking them for input into how my courses and program is impacting them. What information are they using? What didn’t they get from courses that they needed?

I believe this information should be required of all faculty but since it’s not I will start collecting it and providing it for myself for my peace of mind. Hopefully I can start a trend here into better accountability for my own teaching and that of others.

Using students evaluations to evaluate faculty

Posted on by

As my semester gets closer to the end I start receiving emails about end of the semester student evaluations. These evaluations are used to determine if faculty are doing their job (i.e., teaching). So should these be used to evaluate faculty? Well since part of my job in corporate settings was to evaluate training let’s compare what is done in corporate america to determine the success of the course vs what is done in education. In corporate we generally used Kirkpatrick’s evaluation model. As you are reading this keep in mind that student evaluations only measure step 1 of this model. So…

Step 1 – Student reactions. Did students like the training, process, course, instructor? This is the lowest and least important level. It doesn’t tell us if the course, instructor, or training was effective at all. This is the only thing student evaluations measure. Thus faculty are critiqued on whether students liked them or not. Not whether they actually taught anything at all.

Step 2 – Learning. Did the students learn what was taught? This is pretty important. If they learned the content then the course was effective. This is not measured on student evaluations.

Step 3 – Transfer. Are students able to apply what was learned to their jobs? This is very important. This is not measured on student evaluations.

Step 4 – ROI (return on investment). Was the training worth it? What were the benefits of the training? This is extremely important. This is not measured on student evaluations.

As you can see student evaluations are not very important. I mean they tell us whether students liked the course or not. However, thats it. They do not even tell us if students learned anything. Essentially students could rate a course where they learned nothing really high and vice versa making these kind of evaluations worthless when they are by themselves. Thus should faculty be evaluated with these? No. At least not as the only measure and certainly not as an important measure. It’s a pretty worthless measure. There is a reason that we use these other steps in corporate settings – we know that they are meaningful yet for some reason higher education has yet to recognize that.

Essentially what I am trying to say is that if I tried to rate a course or teacher in corporate settings the same way I did in higher ed I would be fired.

Multimedia: Learner Preferences For Multimedia Learning

Posted on by

This is my latest publication. This was a really interesting study on multimedia and I am doing a follow up. Here is the abstract and link to full article:

ABSTRACT: Today’s learners are using multimedia on a daily basis. From computers to cell phones, it’s very difficult to get through a day without being exposed to multimedia. Prior research from Mayer and colleagues has revealed the multimedia principle, which communicates that two representations that explain for one another are better for learning than just one. While much of this research focused on cognitive load and learning, it did not focus on learner preference. As a result, a survey was presented to learners to discover their preferences for the multimedia, modality, redundancy, and coherence principles in a multimedia environment. Overall, participants agreed that they preferred multiple representations to a single one. However, the most surprising results were found when learners were presented redundant representations and irrelevant details. Learners indicated they preferred redundant text and sound with images to image and text or images and sound and that they preferred highly detailed and colored images to simple images. This indicates that while learners may learn better if we following the multimedia principles, they might decrease learner interest or motivation, which could have an impact on instruction

Citation:

Pastore, R. (2014). Multimedia: Learner Preferences For Multimedia Learning. Journal of Multimedia Processing and Technologies, 5(4), 134-144.

Harvard and MIT sued over closed captions on videos and podcasts in online courses

Posted on by

This is something every online instructor needs to consider. Why? Because our materials need to be accessible to everyone. Thus we need to conform to standards, such as 508 when developing our materials. If you were to know more about some of the most sought-after courses, you’d know that the same problem is prevalent within them, too. Obviously, its nearly impossible to make all videos and podcasts with closed captioning because it would take a significant amount of time and resources making these types of media worthless. One of things we have done at UNCW is to make sure that we provide an optional resource that covers similar material. Thus if my course has a podcast I can either type it up, which would take forever, or I can simply provide a reading that contains similar content. For all of my videos, I put them on youtube. Youtube does CC for all of their videos so all of my video sources are usually taken from there. This saves me time and everyone is happy. The only downfall is that my videos are then available to the public and on googles servers, which could be an issue for some people. There are many issues that this lawsuit will bring about such as…should online materials be accessible for everyone if there is a face to face course option? I say that because online courses are definitely not made for everyone (ie people who are not self motivated). Fortunately for me I am not a judge so I do not have make these decisions. I just try my best to make my materials available to everyone. And if you do have a disability or some other issue preventing you from viewing your instructors materials simply ask them for an alternative – I would hope they work with you on this.

You can check out the article here

The MOOC Hype Fades

Posted on by

I have been seeing this trend over the last two years and I predicted this trend when I first heard about moocs. My reason? I have stated it before but one of the comments from this article sums it up nicely so I will repost it. Here is the comment from the article by user hyptiotes

“Remember too that universities have been around since the middle ages. They are resilient institutions. The demise of universities were also claimed with the invention of books (who needs a teacher now!), invention of public libraries (free books to learn at your own pace!), invention of mail order courses, invention of closed television courses, and now the invention of online courses. You can at least understand why academics aren’t worried about their demise (again). Self-motivated learners obviously benefit from MOOCS, just as they did with the invention of public libraries. What keeps universities in business is essentially human laziness. Without a cattle prod and the pedagogical equivalent of a life coach standing over you and checking your work, most people would never finish a course. This is borne out by the abyssmal completion rates of MOOCS. Unless you can cure procrastination, laziness, and minimize cheats in the system for credentials, there will also be a place for credentialed brick-and-mortar universities…”