Research on OER Impact and Effectiveness

Most of the readings set for Week Five dealt with research into the use of OERs. A review was covered in a research shorts video from YouTube entitled A Review of the Effectiveness & Perceptions of Open Educational Resources as Compared to Textbooks (based on Hilton, J. (2016). Open educational resources and college textbook choices: A review of research on efficacy and perceptions. Educational Technology Research and Development, 64(4), 573-590) This review examined 16 empirical studies of courses where OERs had replaced traditional textbooks. The research studies focussed on either the learning objectives and success rates or on the perceptions of students and lecturers regarding OERs. Only one out of the 16 studies found a lower success rate for some students using OERs. The general consensus was that OER use was effective with higher test results and lower rates of failure and/or drop-out. It is noted that researchers appreciated the limitations of their studies and did not attribute direct causality in their conclusions.

Review of Hilton, 2016

The video narrative contained an instance of inaccurate reporting when it stated, “A sizable majority felt that OER were of better quality than traditional textbooks. About half said that they were of similar quality. And only a few thought that OER were inferior”. If you read the original report by Hilton (2016) the statistics are: 33% viewed OER positively, 50% saw OERs and traditional textbooks as of a similar quality, and 17% viewed OERs as inferior. There is no way you can call 33% “a sizable majority”, when a majority implies that it is over half! Still, the results do indicate that both lecturers and students were positive in their perceptions of OER: students valuing the “free” part of OERs, and lecturers appreciating the high quality and flexibility of OERs. My favourite question from Hilton (2016) has to be: “If the average college student spends approximately $1,000 per year on textbooks and yet performs scholastically no better than the student who utilizes free OER, what exactly is being purchased with that $1,000?” This is a question well worth considering.

What is being purchased with that $1,000?

Weller (2012) in a discussion of The openness-creativity cycle in education described the open scholar and the relationship between OERs and creativity. This article comes from the Open University (OU) in the UK. The mission statement from the OU is 'Open to people, places, methods and ideas'. The OU has long been renowned for its open access policies and has been the source of a lot of research into open access.

OU, 1969

Weller (2012) defined a number of related aspects of open education, but the one that was most meaningful to me was the open scholar. The open scholar combines the creation of digital artefacts with a socially oriented distribution network. The open scholar, in my opinion, represents the core of learning in terms of a connectivist pedagogy. I see myself as an open scholar. According to Weller (2012), the open scholar creates, uses, and contributes open content, self-archives, applies his/her own open research, shares, supports open learning initiatives, comments on others, publishes in open access journals, and builds networks. This is the direction in which I have been moving for almost a decade.

Lead educator of SLENZ - my workshop participants

Creativity is high in the list of characteristics of the open scholar. I consider the work I have done creative. I have open research data available on both virtual world projects in which I have taken a lead role. I was lead educator for the foundation build, part of the Second Life Education New Zealand (SLENZ) project. The initial results of this data are available at: Second Life Education in New Zealand: Evaluation Research Final Report The initial research for my literacy game, The Mythical World of Hīnātore, is available here: Literacy game in a virtual world Here are a few pictures of both builds. I have had very positive perceptions from both students and lecturers who have participated in my research. However, the research was not focussed on the aspect of openness but on either the gaming or simulation aspects of the builds. It would be interesting to look further into the impact both builds have made due to the fact they were both created using NZ 3.0 CC licensing. 

Koru SLENZ Foundation build

Literacy game, The Mythical World of Hīnātore

Having taught in an open entry foundation course for nearly a decade, I have always been interested in research into open entry, open access programmes. Dr Barry Hodges from the University of Newcastle in Australia reviewed research into open entry/open access courses (reported in a review I wrote (2017) in Mana Rangahau Issue 1). He concluded that the research evidence indicated that open entry/open access courses suffered from the challenge of four deficits: the challenge of achievement with high drop-out and low success rates; the challenge of academic standards, i.e. accepting failure or lowering standards; the challenge of student support – ““Opportunity without support is not access”; and, the challenge of multiple discourses, acknowledging students who are proficient in discourses other than the dominant. He suggested rejecting the deficit model and creating a dynamic university culture embodying a multiplicity of sub-cultures, each imbued with their own discourses, literacies, and processes.

Me surrounded by my wonderful students

The narrative approach to research must be as important as collecting a mass of statistics. It is incredibly difficult to indicate the effectiveness of an approach to education in terms of causality. There are so many factors that impact on our students and their learning. Switching to an OER may seem to make a positive difference to a group of students. But, it could also be attributed to the motivation of increased digital material and less reliance on the printed page. A number of my own students have conducted research into the use of little OERs (as opposed to big OERs – see Weller, 2012). Some have incorporated OER puzzles, games, and video content into their courses and found the students most receptive to the changes. However, perhaps the positive results obtained could be attributable to their own increased motivation and passion and how these characteristics have been passed on to the students. It is easier to identify causality when examining individual stories and how changes have impacted in particular cases. An interesting opportunity for research has been created by the new government in New Zealand, a result of the election held last month. Our new Prime Minster, Labour leader Jacinda Ardern, has promised that all new tertiary students will receive their first year of tertiary study for free (see the video link below).

Jacinda Ardern, NZ PM (for video link, click the photo)

I imagine a priority for the new Labour Government will be to prove, through research, the benefits of instituting this policy at a national level. However, I tend to be sceptical of politically inspired research and believe that individual institutions should look at the effects in terms of their own students. Stephen Downes, in his video, Research on OER Impact and Effectiveness, made a couple of very important points. He questioned the actual meaning of research and the meaning of impact. Stephen recommends that the information that is needed yet not done, is research into how open resources help society. Yes! OEEs should have a long-term impact on a person’s sense of personal worth and their value to society. In Stephen’s own words: “I see research on grades and graduation rates and course completions and crap like that. And I'm not interested in that. I'm interested in how open resources help society”. I am currently working on my PhD in education, examining the effectiveness of my own literacy game. I have already conducted extensive research using numbers and quantitative data. This has all indicated a positive perception of the game and how it has improved sentence structure and grammar in student’s formal writing. Now, I am looking at narrative, and how the game has personally impacted on select subjects, both short and long-term. The results I have obtained are interesting in that the qualitative data supports the quantitative data. My game may not be a very apt example of seeing something change society but I have seen a game change a mother-son relationship, a son’s progress at school, and a student’s decision-making processes in areas unrelated to the original written literacy in the game. 

Comments on the literacy game - looking back