Concluding the Introduction to K-12 Online Learning Research MOOC

Michael K. Barbour, Wayne State University

As I have with several of the other topics that I have prepared for this MOOC, let me again dip into the material that I wrote for an early draft of an up-coming chapter that will be published sometime in 2013 in the Handbook for Distance Education.

Literature and Research Related to K-12 Online Learning
While the use of K-12 online learning at the K-12 level has been practiced for approximately two decades, the availability of literature and, in particular the published research, to inform that practice has not kept pace. For example, Barbour (2011) reviewed 262 articles from the main distance education journals for Australia, Canada, New Zealand and the United States from 2005 to 2009 (see Table 1).

Table 1. Analysis of K-12 focused articles in the main distance education journals

Australia
Canada
New Zealand
United States
South Africa
American Journal of Distance Education (United States)



8

Distance Education (Australia)
2


4
2
Journal of Distance Education (Canada)
1
4



Journal of Distance Learning (New Zealand)

1.5*
1
.5*

Total
3
5.5
1
12.5
2
*One article had a focus on both Canada and the United States

As indicated in Table 3, only 24 articles (or less than 10%) related to K-12 distance education<

However, this is changing. Cavanaugh, Barbour and Clark (2009) indicated that of the 226 publications included in their review of the literature there were only 29 items were from 1997 to 2000, but there were 69 items published from 2006 to 2009. To date there have been three major reviews of the literature related to K-12 online learning that have been published. The first was Rice (2006), who conducted a review of the K-12 distance education literature from 1995 to 2005 (although the review primarily focused on literature from 2000 to 2005). Second, Barbour and Reeves (2009) examined literature related to K-12 online learning from 1994 to 2008. Finally, Cavanaugh, Barbour and Clark (2009) examined only the open access literature from 1997 to 2008.

Literature on K-12 Online Learning
In their review of the open access literature, Cavanaugh et al. (2009) stated that the published literature related to K-12 online learning was primarily “based upon the personal experiences of those involved in the practice of virtual schooling” (¶ 5). They further indicated this literature was largely based on focused upon the opinions and/or experiences of those involved with K-12 online learning, such as online teachers or administrators of online programs. Within this largely practitioner-focused literature, Cavanaugh et al. describe it as being mainly literature about “statewide and consortium/multi-district virtual schools, the roles of teachers and administrators, the promise of virtual schooling and its initial rationale for implementation, administrative challenges, the technology utilized, and interact with students” (Conclusions and Implications, ¶ 1).

In their book Virtual Schools: Planning for Success, Berge and Clark (2005) described five potential benefits of and five challenges facing to K-12 online learning.
  • Potential benefits: higher levels of motivation; expanding educational access; providing high-quality learning opportunities; improving student outcomes and skills; allowing for educational choice; and administrative efficiency
  • Challenges facing: high start-up costs associated with virtual schools; access issues surrounding the digital divide; approval or accreditation of virtual schools; and student readiness issues and retention issues
In their review of the literature, Barbour and Reeves (2009) used these potential benefits and challenges facing K-12 online learning to classify the existing literature (see Tables 2 and 3 ).

Table 2. Summary of the benefits of virtual schooling (Barbour & Reeves, 2009, p. 409)
Benefit
Reference
Higher levels of motivation
Kellogg & Politoski (2002)
Expanding educational access
Berge & Clark (2005); Cavanaugh (2001); Freedman, Darrow, Watson & Lorenzo (2002); Fulton (2002); Hernandez (2005); Kellogg & Politoski (2002); Zucker (2005)
Providing high-quality learning opportunities
Berge & Clark (2005); Butz (2004); Elbaum & Tinker (1997); Fulton (2002); Kaplan-Leiserson (2003); Kellogg & Politoski (2002); Thomas (1999; 2000; 2003); Tinker & Haavind (1997)
Improving student outcomes and skills
Berge & Clark (2005); Zucker & Kozma (2003)
Allowing for educational choice
Baker, Bouras, Hartwig & McNair (2005); Berge & Clark (2005); Butz (2004); Fulton (2002); Hassell & Terrell (2004)
Administrative efficiency
Keeler (2003); Russo (2001); Vail (2001)
Table 3. Summary of the challenges of virtual schooling (Barbour & Reeves, 2009, p. 411)
Challenge
Reference
High start-up costs associated with virtual schools
Berge & Clark (2005); Morris (2002)
Access issues surrounding the digital divide
Berge & Clark (2005)
Approval or accreditation of virtual schools
Berge & Clark (2005)
Student readiness issues and retention issues
Ballas & Belyk (2000); Barker & Wendel (2001); Berge and Clark (2005); Bigbie & McCarroll (2000); Cavanuagh, Gillan, Bosnick, Hess & Scott (2005); Clark, Lewis, Oyer, & Schreiber (2002); Espinoza, Dove, Zucker & Kozma (1999); Haughey & Muirhead (1999); Kozma, Zucker & Espinoza (1998); McLeod, Hughes, Brown, Choi & Maeda (2005); Zucker & Kozma (2003)
It is important to point out, as Barbour and Reeves did themselves, that the benefits listed were only potential benefits. As Barbour (2010) further underlined:
[Barbour and Reeves] were careful to remind readers that while online learning may allow for educational improvements such as a high levels of learner motivation, high quality learning opportunities or improvement in student outcomes, it certainly did not guarantee any of these potential benefits would be realized simply by the introduction of online learning. (p. 7)

As the research to date has clearly shown, none of these potential benefits have been proven by empirical studies using reliable and valid methodology.

Research on K-12 Online Learning
It is important for those in the field of K-12 online learning to distinguish between the published literature in the field and the literature that is actually based upon research. While there is a growing body of literature, Barbour and Reeves (2009) wrote that “there [had] been a deficit of rigorous reviews of the literature related to virtual schools” (p. 402), and that “much of the research [was] only available in unpublished Master’s theses and Doctoral dissertations” (p. 403). This was similar to the conclusions draw by DiPietro, Ferdig, Black and Preston (2008), who stated “research-based investigations into the teaching and learning process in this medium and at this level are still lacking” (p. 10). Finally, Rice (2006) was likely the most direct in her assessment of the research into K-12 online learning when she lamented that “a paucity of research exists when examining high school students enrolled in virtual schools, and the research base is smaller still when the population of students is further narrowed to the elementary grades” (p. 430). However, Cavanaugh et al. (2009) indicated:
  • in many ways, this [was] indicative of the foundational descriptive work that often precedes experimentation in any scientific field. In other words, it is important to know how students in virtual school engage in their learning in this environment prior to conducting any rigorous examination of virtual schooling. (Literature Review, ¶ 1)

While this may be the case, given that K-12 online learning is now two decades old, has more than 4 million students enrolled in one or more courses, and has been adopted in the United States by the educational reform movement (Bush & Wise, 2010); shouldn’t we expect there to be more evidence of the success of K-12 online learning and, in particular, the factors that ensure student success in the online environment.

There is general agreement about the themes that have been dominant in the limited amount of research conducted on K-12 online learning to date. Rice (2006) described the research into K-12 online learning as either being comparisons of student performance between those enrolled in online and face-to-face environments or examinations of the qualities and characteristics of the online learning experience; with the comparative research being the dominant of the two groups. Similarly, Cavanaugh et al. (2009) also indicated that the research into K-12 online learning fell into two categories: effectiveness and issues related to student readiness and retention. Cavanaugh and her colleagues also indicated that the majority of the research had focused on the effectiveness category

An examination of the findings related to comparison of student performance in K-12 online learning environments and the traditional classroom has been mixed (see Table 4).

Table 4. Summary of research related to the effectiveness of K-12 online learning
Study
Finding
Ballas & Belyk (2000)
performance of virtual and classroom students in Alberta were similar in English and Social Studies courses, but that classroom students performed better overall in all other subject areas
Bigbie & McCarroll (2000)
over half of the students who completed FLVS courses scored an A in their course and only 7% received a failing grade
Barker & Wendel (2001)
students in the six virtual schools in three different provinces performed no worse than the students from the three conventional schools
Cavanaugh et al. (2005)
FLVS students performed better on a non-mandatory assessment tool than students from the traditional classroom
McLeod et al. (2005)
FLVS students performed better on an assessment of algebraic understanding than their classroom counterparts
Barbour & Mulcahy (2008)
little difference in the overall performance of students based upon delivery model
Barbour & Mulcahy (2009)
no difference in student performance based upon method of course delivery
However, these general findings do not tell the complete story. For example, Mulcahy and Barbour (2010) later speculated that weaker students may have been self-selecting a less rigorous curriculum in order to avoid taking online courses (a finding that was also supported by Mulcahy, Dibbon and Norberg [2008]). This kind of skewing of the potential sample from the K-12 online learning is quite common in the studies listed above (see Table 5).

Table 5. Methodological issues with the K-12 online learning samples in comparative studies
Study
Sample
Ballas & Belyk (2000)
participation rate in the assessment among virtual students ranged from 65% to 75% compared to 90% to 96% for the classroom-based students
Bigbie & McCarroll (2000)
between 25% and 50% of students had dropped out of their FLVS courses over the previous two-year period
Cavanaugh et al. (2005)
speculated that the virtual school students who did take the assessment may have been more academically motivated and naturally higher achieving students
McLeod et al. (2005)
results of the student performance were due to the high dropout rate in virtual school courses
This kind of student selectivity in the K-12 online learning samples should not be surprising to anyone familiar with its practice. With the exception of the past three to five years, the literature related to K-12 online learning has provided a fairly consistent description of K-12 online learners (see Table 6).

Table 6. Description of K-12 online learner from the research
Study
Sample
Kozma et al. (1998)
vast majority of VHS students in their courses were planning to attend a four-year college
Espinoza et al., 1999
VHS courses are predominantly designated as ‘honors,’ and students enrolled are mostly college bound
Haughey & Muirhead (1999)
preferred characteristics include the highly motivated, self-directed, self-disciplined, independent learner who could read and write well, and who also had a strong interest in or ability with technology
Roblyer & Elbaum (2000)
only students with a high need to control and structure their own learning may choose distance formats freely
Clark et al. (2002)
IVHS students were highly motivated, high achieving, self-directed and/or who liked to work independently
Mills (2003)
typical online student was an A or B student
Watkins (2005)
45% of the students who participated in e-learning opportunities in Michigan were either advanced placement or academically advanced students
Rice (2006) summarized this problem by indicating the research into the effectiveness of K-12 online learning as being “challenged with issues of small sample size, dissimilar comparison groups, and differences in instructor experience and training” (p. 431, emphasis added). She concluded “that the effectiveness of distance education appears to have more to do with who is teaching, who is learning, and how that learning is accomplished, and less to do with the medium” (p. 440, emphasis added).

While Cathy Cavanaugh, in her chapter in this handbook, concludes that the research into the effectiveness of K-12 online learning “suggests that as distance education is currently practiced, student learning on average in well-designed online elementary and secondary environments appears to be equivalent to learning in a well-designed classroom environment” (p. # still to be assigned). The potential problems with the K-12 online learning samples from the studies included in those meta-analyses should call into question that conclusion.[1] This is particularly true when some have indicated that there is a growing segment of K-12 online learning students who would fall into the category of at-risk students (Barbour, 2009; 2011; Klein, 2006; Rapp, Eckes & Plurker, 2006; Watson, Gemin & Ryan, 2008).


[1] For additional complications related to the conclusions drawn from meta-analysis, see Hattie (2009) and Hattie and Marsh (1996, 2006).


References

Baker, J. D., Bouras, C., Hartwig, S. M., & McNair, E. R. (2005). K12, Inc. and the Colorado Virtual Academy: A virtual charter school. In Z. L. Berge & T. Clark (Eds.), Virtual schools: Planning for success (pp. 133-142). New York: Teachers College Press.

Ballas, F. A., & Belyk, D. (2000). Student achievement and performance levels in online education research study. Red Deer, AB: Schollie Research & Consulting. Retrieved from http://web.archive.org/web/20051031044348/http:www.ataoc.ca/files/pdf/AOCresearch_full_report.pdf

Barbour, M. K. (2009). Today's student and virtual schooling: The reality, the challenges, the promise... Journal of Distance Learning, 13(1), 5-25.

Barbour, M. K. (2010). Researching K-12 online learning: What do we know and what should we examine? Distance Learning, 7(2), 7-12.

Barbour, M. K. (2011). The promise and the reality: Exploring virtual schooling in rural jurisdictions. Education in Rural Australia, 21(1), 1-20.

Barbour, M. K., & Mulcahy, D. (2008). How are they doing? Examining student achievement in virtual schooling. Education in Rural Australia, 18(2), 63-74.

Barbour, M. K., & Mulcahy, D. (2009). Student performance in virtual schooling: Looking beyond the numbers. ERS Spectrum, 27(1), 23-30.

Barbour, M. K., & Reeves, T. C. (2009). The reality of virtual schools: A review of the literature. Computers and Education, 52(2), 402–416.

Barker, K., Wendel, T., & Richmond, M. (1999). Linking the literature: School effectiveness and virtual schools. Vancouver, BC: FuturEd. Retrieved from http://web.archive.org/web/20061112102653/http:www.futured.com/pdf/Virtual.pdf

Berge, Z. L., & Clark, T. (2005). Virtual schools: Planning for success. New York, NY: Teachers College Press.

Bigbie, C., & McCarroll, W. (2000). The Florida high school evaluation 1999-2000 report. Tallahassee, FL: Florida State University.

Bush, J. & Wise, B. (2010). Digital learning now. Tallahassee, FL: Foundation for Excellence in Education. Retrieved from http://www.excelined.org/Docs/Digital Learning Now Report FINAL.pdf

Butz, C. (2004). Parent and student satisfaction with online education at the elementary and secondary levels. Unpublished Dissertation, University of Nevada at Las Vegas, Las Vegas, NV.

Cavanaugh, C. (2001). The effectiveness of interactive distance education technologies in K-12 learning: A meta-analysis. International Journal of Educational Telecommunications, 7(1), 73-88.

Cavanaugh, C., Barbour, M. K., & Clark, T. (2009). Research and practice in K-12 online learning: A review of literature. International Review of Research in Open and Distance Learning, 10(1). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/607

Cavanaugh, C., Gillan, K. J., Bosnick, J., Hess, M., & Scott, H. (2005). Succeeding at the gateway: Secondary algebra learning in the virtual school. Jacksonville, FL: University of North Florida.

Clark, T., Lewis, E., Oyer, E., & Schreiber, J. (2002). Illinois Virtual High School Evaluation, 2001-2002. Carbondale, IL: TA Consulting and Southern Illinois University. Retrieved from http://www2.imsa.edu/programs/ivhs/pdfs/IVHS_FinalRpt.pdf

DiPietro, M., Ferdig, R. E., Black, E. W. & Preston, M. (2008). Best practices in teaching k–12 online: Lessons learned from Michigan Virtual School teachers. Journal of Interactive Online Learning, 7(1). Retrieved from http://www.ncolr.org/jiol/issues/getfile.cfm?volID=7&IssueID=22&ArticleID=113

Elbaum, B., & Tinker, R. (1997). A review of secondary netcourses and virtual schools. Concord, MA: Concord Consortium.

Espinoza, C., Dove, T., Zucker, A., & Kozma, R. (1999). An evaluation of the Virtual High School after two years in operation. Arlington, VA: SRI International. Retrieved from http://ctl.sri.com/publications/downloads/evalvhs2yrs.pdf

Freedman, G., Darrow, R., Watson, J., & Lorenzo, G. (2002). California virtual school report: A national survey of virtual education practice and policy with recommendations for the State of California. Lorenzo Associates, Inc. Retrieved from http://web.archive.org/web/20061207213241/http:www.edpath.com/images/VHSReport.pdf

Fulton, K. (2002). Preserving principles of public education in an online world. Washington, DC: Center on Education Policy. Retrieved from http://web.archive.org/web/20060524121739/http:www.ctredpol.org/democracypublicschools/preserving_principles_online_world_full.pdf

Hassell, B. C., & Terrell, M. G. (2004). How can virtual schools be a vibrant part of meeting the choice provisions of the No Child Left Behind Act?. Virtual School Report. Retrieved from http://www2.ed.gov/about/offices/list/os/technology/plan/2004/site/documents/Hassel-Terrell-VirtualSchools.pdf

Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analysis related to achievement. New York: Routledge.

Hattie, J., & Marsh, H. W. (1996). The relationship between research and teaching: A meta-analysis. Review of Educational Research, 66, 507–542.

Hattie, M., & Marsh, J. (2002). The relationship between productivity and teaching effectiveness. Journal of Higher Education, 73(5), 603-641.

Haughey, M., & Muirhead, W. (1999). On-line learning: Best practices for Alberta school jurisdictions. Edmonton, AB: Government of Alberta. Retrieved from http://www.phrd.ab.ca/technology/best_practices/on-line-learning.pdf

Hernandez, F. J. (2005). Equity and access: The promise of virtual schools. In Z. L. Berge & T. Clark (Eds.), Virtual schools: Planning for success (pp. 20-34). New York, NY: Teachers College Press.

Kaplan-Leiserson, E. (2003). We-learning: Social software and e-learning. Learning Circuits. Retrieved from http://www.astd.org/LC/2003/1203_kaplan.htm

Keeler, C. (2003). Developing and using an instrument to describe instructional design elements of high school online courses. Unpublished Dissertation, University of Oregon, Eugene, OR.

Kellogg, L., & Politoski, K. (2002). Virtual schools across America: Trends in K-12 online education. Los Angeles, CA: Peak Group Research Corporation.

Klein, C. (2006). Virtual charter schools and home schooling. Youngston, NY: Cambria Press.

Kozma, R., Zucker, A., & Espinoza, C. (1998). An evaluation of the Virtual High School after one year in operation. Arlington, VA: SRI International. Retrieved from http://ctl.sri.com/publications/downloads/evalvhs1yr.pdf

McLeod, S., Hughes, J. E., Brown, R., Choi, J., & Maeda, Y. (2005). Algebra achievement in virtual and traditional schools. Naperville, IL: Learning Point Associates.

Mills, S. (2003). Implementing online secondary education: An evaluation of a virtual high school. In C. Crawford et al. (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2003 (pp. 444-451). Chesapeake, VA: AACE.

Morris, S. (2002). Teaching and learning online: A step-by-step guide for designing an online K-12 school program. Lanham, MD: Scarecrow Press Inc.

Mulcahy, D., & Barbour, M. K. (2010, May). Duck and cover: Are rural students taking basic courses to avoid taking them online? A roundtable presented at the annual meeting of the American Educational Research Association, Denver, CO.

Mulcahy, D. M., Dibbon, D., & Norberg, C. (2008). An investigation into the nature of education in a rural and remote region of Newfoundland and Labrador: The Straits. St. John’s, NL: The Harris Centre, Memorial University of Newfoundland.

Rapp, K. E., Eckes, S. E., & Plurker, J. A. (2006). Cyber charter schools in Indiana: Policy implications of the current statutory language. Education Policy Brief, 4(3). (online) Retrieved from http://ceep.indiana.edu/projects/PDF/PB_V4N3_Winter_2006_CyberCharter.pdf

Rice, K. L. (2006). A comprehensive look at distance education in the K-12 context. Journal of Research on Technology in Education, 38(4), 425-448.

Roblyer, M. D., & Elbaum, B. (2000). Virtual learning? Research on virtual high schools. Learning & Leading with Technology, 27(4), 58-61.

Russo, A. (2001). E-learning everywhere. The School Administrator. Retrieved from http://www.aasa.org/SchoolAdministratorArticle.aspx?id=10698

Thomas, W. R. (1999). Essential elements for web-based courses for high school students. Atlanta, GA: Southern Regional Education Board. Retrieved from http://publications.sreb.org/1999/99T03_EssentialElements.pdf

Thomas, W. R. (2000). Essential principles of quality: Guidelines for web-based courses for middle and high schools. Atlanta, GA: Southern Regional Education Board. Retrieved from http://publications.sreb.org/2000/00T04_EssentialPrinciples.pdf

Thomas, W. R. (2003). Essential principles of high-quality online teaching: Guidelines for evaluating K-12 online teachers. Atlanta, GA: Southern Regional Education Board. Retrieved from http://publications.sreb.org/2003/03T02_Essential_Principles.pdf

Tinker, R., & Haavind, S. (1997). Netcourses and netseminars: Current practice and new designs. Concord, MA: Concord Consortium. Retrieved from http://archive.concord.org/publications/pdf/netcours.pdf

Vail, K. (2001). Online learning grows up: No longer an experiment, virtual school is here to stay. Electronic School. Retrieved from http://web.archive.org/web/20080608145221/http:www.electronic-school.com/2001/09/0901f1.html

Watkins, T. (2005). Exploring e-learning reforms for Michigan: The new educational (r)evolution. Detroit, MI: Wayne State University. Retrieved from http://web.archive.org/web/20051208000848/http:www.coe.wayne.edu/e-learningReport.pdf

Watson, J. F., Gemin, B., Ryan, J., & Wicks, M. (2009). Keeping pace with K–12 online learning: A review of state-level policy and practice. Evergreen, CO: Evergreen Education Group. Retrieved from http://www.kpk12.com/downloads/KeepingPace09-fullreport.pdf

Zucker, A. (2005). A study of student interaction and collaboration in the Virtual High School. Naperville, IL: Learning Point Associates.

Zucker, A., & Kozma, R. (2003). The Virtual High School: Teaching generation V. New York, NY: Teachers College Press.

Readings:

No assigned reading.

Resources:

Blogging Activity


Post a blog entry that responds to one of the following prompts:
  1. The comparative literature references above focuses solely on the supplemental K-12 online learning environment. Locate one research article that focuses specifically on comparing student performance in the full-time K-12 online learning environment with student performance in the traditional brick-and-mortar environment and critique that study.
    OR
  2. Identify a piece of empirical research into K-12 online learning (ideally a dissertation study from the ProQuest Dissertation Database). Describe and then critique that study.