Learning as a dependent variable in experimental research

I am reading through a number of peer reviewed studies and noticing that there are pieces missing from each. So if you are conducting an experimental study where learning is a dependent variable you must do the the following (note these are the things I keep seeing that are missing):

  1. Pre test. This is to ensure that differences found in your study are due to the condition and not prior knowledge. Be aware that if you give the same pre test as post test, its a violation of validity but it can still be done (its better than no pre test). Thus I recommend giving a different pre test.
  2. Content Validity. Is the content you are using correct? And is the assessment measuring the objectives from the content? How do you know this? Explain this. Has an expert reviewed it? Have you piloted it?
  3. What is the reliability of the test? Please provide us this information.
  4. Provide the M, SD, and Effect.

I know all of this seems like common sense but I am seeing top tier journals publishing articles that do not contain this basic information.

A systematic literature review of empirical evidence on computer games and serious games

A very good article in Computers and Education which analyzes the empirical evidence from the gaming and learning literature:

Connolly, T., M., Boyle, E., A., MacArthur, E., Hainey, T., & Boyle, J., M. (2012). A systematic literature review of empirical evidence on computer games and serious games. Computers and Education, 59(2), 661-686

Abstract

This paper examines the literature on computer games and serious games in regard to the potential positive impacts of gaming on users aged 14 years or above, especially with respect to learning, skill enhancement and engagement. Search terms identified 129 papers reporting empirical evidence about the impacts and outcomes of computer games and serious games with respect to learning and engagement and a multidimensional approach to categorizing games was developed. The findings revealed that playing computer games is linked to a range of perceptual, cognitive, behavioural, affective and motivational impacts and outcomes. The most frequently occurring outcomes and impacts were knowledge acquisition/content understanding and affective and motivational outcomes. The range of indicators and measures used in the included papers are discussed, together with methodological limitations and recommendations for further work in this area.

The effects of time-compressed instruction and redundancy on learning and learners’ perceptions of cognitive load

My recent article published in Computers and Education:

Abstract: Can increasing the speed of audio narration in multimedia instruction decrease training time and still maintain learning? The purpose of this study was to examine the effects of time-compressed instruction and redundancy on learning and learners’ perceptions of cognitive load. 154 university students were placed into conditions that consisted of time-compression (0%, 25%, or 50%) and redundancy (redundant text and narration or narration only). Participants were presented with multimedia instruction on the human heart and its parts then given factual and problem solving knowledge tests, a cognitive load measure, and a review behavior (back and replay buttons) measure. Results of the study indicated that participants who were presented 0% and 25% compression obtained similar scores on both the factual and problem solving measures. Additionally, they indicated similar levels of cognitive load. Participants who were presented redundant instruction were not able to perform as well as participants presented non-redundant instruction.

http://www.sciencedirect.com/science/article/pii/S0360131511002351