evaluation – Dr. Ray Pastore's Instructional Technology Blog

Evaluation for an instructor led course (the materials)

admin — Fri, 13 Jun 2014 12:59:04 +0000

Evaluation for an instructor led course (the materials)

The following is a guide for evaluating materials for an instructor led course. This is not a guide for evaluating a person teaching the course.

Criteria	Description	Scale
Criteria	Description		N/A	Comments
Content
Working memory/Cognitive load	Learners can hold 5-7 concepts in their working memory at one time. Does this content ask learners to work with more at one time (per screen/topic)? Thus screens should have limited text/bullet points.
Multimedia principles	Images and text (or narration) that explain for one another are better for learning and communication then just one by itself. Thus PPT or lecture should follow this model.
No extra details – only what is required	Extra images, text, or colors distracts
Highlight important information	Use of cues or color coding for important concepts
No busy screens/backgrounds/fonts	These distract learners
Objectives are clear	What is the point?
Content is inline with objectives	Does the site do what it says?
Assessment is inline with content	Is there an assessment and does it asses each objective?
Flow	Introduction, content, ending
Learning strategies	Are learning strategies apparent and appropriate?
Motivation	Are motivation (ARCS) applied?
Learners apply/problem solve (ie case studies)	Are the learners actually learning more than factual information?
Sources	To back up claims
Assessment
Is there an assessment?
Assessment measures high level knowledge	Does the assessment measure more than factual information?
Is there a satisfaction survey?
Style
Colors/fonts/layout	These should be standard throughout
Colors	Should NOT be more than 2-4 colors on screen
fonts	Are fonts legible
layout	Grid and organized
Simple and natural dialogue	Speak the users’ language – informal is better than formal for learning
Rule of thirds	Does the screen utilize them?
Technical
Audio	Sound/voices volume, background noise, easy to understand
File formats	Are they compatible with all hardware
Video	Does it play well/take a while to load?
Develop for the output technology	Resolution – is the resolution look correct or pixelated?, colors look right on screen, golden ratio/rectangle (organization and layout look correct)
Instructor Guide
Time	How long does it take to complete each section?
Narration	Does it tell the instructor what to say?
Notes	Are there notes for the instructor?
Materials	Are all materials provided for the instructor?
Facilitating/teaching	Does the instructor guide contain enough information that one could pick it up and teach the course (assuming they have all the materials, know the content, know how to teach)?
Student Guide
Schedule	Do students know the schedule of the course and topics?
Objectives	Do students have the objectives?
Content	Do students have a copy of the PPT/Materials to take notes and follow along?
Materials	Are student materials included in this guide?
Job aid	Is there a job aid for students to take with them to help them transfer knowledge from course to job?

Usability Evaluation for websites and computer based instruction

admin — Wed, 11 Jun 2014 22:30:07 +0000

Usability Evaluation for websites and computer based instruction

The following is a guide for evaluating computer based instruction and websites. It can also be used for mobile and tablet based software. This will vary based on audience and objectives. Some criteria may or may not apply to all devices or instructional media types. This is a general evaluation and should be modified for every single project. Keep in mind I developed this with training/learning in mind.

Criteria	Description	Scale
Criteria	Description		N/A	Comments
Content
Working memory/Cognitive load	Learners can hold 5-7 concepts in their working memory at one time. Does this content ask learners to work with more at one time (per screen/topic)? Thus screens should have limited text/bullet points.
Multimedia principles	Images and text (or narration) that explain for one another are better for learning and communication then just one by itself
No extra details – only what is required	Extra images, text, or colors distracts
Highlight important information	Use of cues or color coding for important concepts
Objectives are clear	What is the point?
Content is inline with objectives	Does the site do what it says?
Assessment is inline with content	Is there an assessment and does it asses each objective?
Flow	Introduction, content, ending
Learning strategies	Are learning strategies apparent and appropriate?
Learners apply/problem solve	Are the learners actually learning more than factual information?
Sources	To back up claims
Assessment
Is there an assessment?
Assessment measures high level knowledge	Does the assessment measure more than factual information?
Is there a satisfaction survey?
Style
Fitt’s law	Buttons should be big and close enough to prevent error
Colors/fonts/layout	These should be standard throughout
No busy screens/backgrounds/fonts	These distract learners
Colors	Should NOT be more than 2-4 colors on screen
fonts	Are fonts legible
layout	Grid and organized
Simple and natural dialogue	Speak the users’ language – informal is better than formal for learning
User friendly layout	Are there directions on using the tutorial?
Rule of thirds	Does the screen utilize them?
Functionality
Provide feedback	Feedback for navigation
Provide clearly marked exits	Get to home screen/Exit program
Provide shortcuts	Breadcrumbs
Reversal of actions	Can you go back?
Errors	Deal with errors in positive and helpful manner
Provide help and documentation	Help button/documentation
Navigation	Menu, next, back, help buttons – easy to find
Technical
Load time	Does it load in a few seconds?
Develop for the output technology	Resolution – is the resolution look correct or pixelated?, colors look right on screen, golden ratio/rectangle (organization and layout look correct)
Video	Does it play well/take a while to load?
Audio	Sound/voices volume, background noise, easy to understand
File formats	Are they compatible with all hardware
508 Compliance	Does it meet 508? i.e. alt tags
SCORM Compliance	For an LMS