National study: Students fall behind a year in math in online charter schools

A few years back, the state Legislature, enthralled by what seemed the endless possibilities of virtual education, considered a bill that would have mandated every high school student in Georgia complete at least one course online.

At the time, cyber learning was being heralded as an innovative solution to teach disengaged students at a lower cost than brick and mortar schools.

NO CAPTION

NO CAPTION

A series of studies have found limits to the effectiveness of online education, saying it works best yoked to classroom instruction, and for a limited band of students, the motivated and disciplined.

The latest and most devastating set of reservations about virtual learning comes from the first national study of academic attainment in the nation’s public online charter schools. Conducted by three independent research groups, the National Study of Online Charter Schools was funded by a strong advocate of charter schools, the Walton Family Foundation.

The Center for Research on Education Outcomes or CREDO at Stanford University, the Center on Reinventing Public Education at the University of Washington and Mathematica Policy Research looked at students in 158 virtual schools in 17 states including Georgia. The students took all their classes online. Most of these schools were run by private providers rather than school districts.

The findings were chilling. In math, students were 180 days – an entire school year – behind peers in traditional public schools. In reading, they were 72 days behind. Overall, 88 percent of online charters posted lower math growth than comparable standard public schools.

A bright spot for Georgia: Online charter students here and in Wisconsin showed reading gains higher than peers in traditional public schools.  The CREDO study noted, “These findings show it is possible for online charter schools to produce stronger growth, but it is not the common outcome.”

The dismal national math finding led project director Margaret Raymond to tell the Washington Post: “There’s still some possibility that there’s positive learning, but it’s so statistically significantly different from the average, it is literally as if the kid did not go to school for an entire year.”

In response to the findings that online charter school students had such lower academic performance, charter operators have been critical of the project methods.

But Walton welcomed the results. “We support research on difficult questions like these because we want to know what is working for kids — and what is not. Innovation in education takes time, but we must test whether new ideas are working and make changes when we learn that ideas with potential are falling short,” said the director of research and evaluation for the Walton Family Foundation, Marc Holley. “We’re grateful that CRPE, Mathematica, and CREDO have studied these schools and are sharing their findings. Knowing the facts helps parents, educators, policymakers, and funders make smarter, more informed decisions that benefit children.”

Cyber classes still seem to work best for the first generation of virtual learners: the ambitious and determined teens who sought out online options to take the honors and AP courses their brick-and-mortar schools didn’t offer. These are school-proof students who thrive in any learning environment and are highly self-directed.

In the wake of the findings from the Walton-funded project, Georgia ought to look at its online charter schools. We need to know which are failing and which are succeeding. Expand the successful ones and close the failures where kids lose an entire year of math. While many folks on this blog blast traditional public schools, the learning loss in low-performing brick and mortar schools is typically described in weeks, not years.

Here are some of the official findings:

Conducted by three independent research institutions, the study is the most comprehensive examination of online charter schools to date, and is organized into separate, topical report volumes.

Mathematica’s report offers a snapshot of the 200 online charter schools operating across the country and the 200,000 elementary, middle, and high school students they serve. The report examines the instructional programs of online charter schools; methods used to engage students and parents, along with expectations of parental involvement; the teachers and principals of online charter schools; and the schools’ management and governance. Mathematica’s analysis finds:

-Student–driven, independent study is the dominant mode of learning in online charter schools, with 33 percent of online charter schools offering only self-paced instruction.

-Online charter schools typically provide students with less live teacher contact time in a week than students in conventional schools have in a day.

-Maintaining student engagement in this environment of limited student-teacher interaction is considered the greatest challenge by far, identified by online charter school principals nearly three times as often as any other challenge.

-Online charter schools place significant expectations on parents, perhaps to compensate for limited student-teacher interaction, with 43, 56, and 78 percent of online charters at the high school, middle, and elementary grade levels, respectively, expecting parents to actively participate in student instruction Brian Gill, a Mathematica senior fellow and lead author of the report, said, “Challenges in maintaining student engagement are inherent in online instruction, and they are exacerbated by high student-teacher ratios and minimal student-teacher contact time, which the data reveal are typical of online charter schools nationwide. These findings suggest reason for concern about whether the sector is likely to be effective in promoting student achievement.”

The Center on Reinventing Public Education conducted an extensive examination of how state policy shapes the online charter school landscape. Researchers found that online charter schools exist in a number of different policy environments due to variation in state charter law and administrative regulation. Most of the existing regulation is reactive to controversy (restrictions on growth and autonomy), rather than proactive policies to guide the unique opportunities and challenges of online charters. The authors found several drawbacks to forcing online schools into the charter context, including:

– Open admission requirements that prevent schools from screening for students who are most likely to be successful in an online school.

– Authorizing and accountability provisions that are not well suited to the unique challenges of regulating online schools.

-Funding mechanisms that preclude outcomes-based funding CRPE director Robin Lake, who co-authored the study, said, “We need policies that address legitimate concerns without needlessly restricting growth.” The report recommends that policymakers consider moving online schools out of the charter context, or craft unique provisions specific to online charters.”

The CREDO at Stanford University report presents the most comprehensive findings available to date about impacts of online charter enrollment on the academic progress of students. While findings vary for each student, the results in CREDO’s report show that the majority of online charter students had far weaker academic growth in both math and reading compared to their traditional public school peers.

To conceptualize this shortfall, it would equate to a student losing 72 days of learning in reading and 180 days of learning in math, based on a 180-day school year. This pattern of weaker growth remained consistent across racial-ethnic subpopulations and students in poverty.

“While the overall findings of our analysis are somber, we do believe the information will serve as the foundation for constructive discussions on the role of online schools in the K-12 sector. We see an opportunity for the providers to do a better job of documenting the benefits they provide to their students and to connect with and learn from operators who are doing well, “said Dr. James Woodworth, Senior Quantitative Research Analyst for CREDO at Stanford University.

Reader Comments 0

29 comments
Wascatlady
Wascatlady

I had to push in to a high school alternative school to provide ESOL assistance in our local system and I found it an absolute travesty in terms of instruction.  It was student-led,on-line, and it was bull hockey.  The kids would skim the instruction, then go quickly to the test, which they would fail.  Then they would ask the teacher (monitor) to reset the test, and they would take it again, until they finally got the required score.  They no more learned anything (except gaming the system) than a man in the moon.


These kids, who needed the most diagnosis, the most true instruction, the most monitoring and direction, the most discipline, got none.  I think the program they were using is widely used for computer-based "instruction."  

smithmc
smithmc

These results confirm the obvious.  Whoever thought sitting at a computer could replicate, replace or improve on human interaction in a classroom?  At best, online support can enhance the classroom experience, but it cannot replace it.  Except for the very motivated.  


Duh?

class80olddog
class80olddog

Typical post. Charters bad. Traditional schools good. Parents want charters. Parents stupid. Teachers smart.

Wascatlady
Wascatlady

@class80olddog As I understand it, this study data- matched computer-charter to traditional, and computer-charter to charter, NOT charter to traditional. Therefore, it came to no conclusions about charter vs traditional.  It focused on how well, or not, computer-led instruction worked for these "virtual, on-line schools."

class80olddog
class80olddog

I think deep, just not PC. For example, group SPED students in one room where they can be taught at their precise instructional level.

dreema
dreema

@class80olddog Didn't read the article did you? It cites actual research by multiple organizations, and points out that online instruction works best for motivated students who would learn anyway. 

HS_Math_Teacher
HS_Math_Teacher

There are some kids who can learn math from reading a textbook, and working the problems at the end of each section - without a teacher to explain or demonstrate anything.  I suppose these type of kids are the ones who can excel in an online math course.  I remember out of necessity doing this in a college course, in that I couldn't understand the professor's broken English. 

CSpinks
CSpinks

That The Walton Family Foundation employed independent evaluators to determine the worth of on-line Math instruction is encouraging.

MaryElizabethSings
MaryElizabethSings

@CSpinks 

The point is:  Why have the Walton family (and Bill Gates, for that matter) set themselves up as ambassadors to improve public education?  Money never has equated to knowledge.

MaryElizabethSings
MaryElizabethSings

@EdJohnson @MaryElizabethSings @CSpinks 

Well understood and stated article, Ed.  Thank you for sharing it here.  I must cut and paste one paragraph from this article in order to highlight the thoughts, therein:


"For Bill Gates this has all been a grand experiment, one that he believes he is entitled to conduct on our children, our teachers and our schools. It is astounding that a man, who has no qualifications to guide our nation’s educational system has been allowed, by virtue of his fortune, to meddle in it as he has."

I would like to add, "Ditto, the Walton family."

 

CSpinks
CSpinks

Should anyone be surprised that Math teachers are more effective instructors than are computers?

eulb
eulb

My experience with online instruction matches the description in this article.   My son was a student in an online virtual school for his 6th grade year.  Our family stuck it out all year, but we were very disappointed in it and did not re-enroll him the following year.  I was satisfied with my son's literature and history assignments.  He retained some of that info.  Other subjects -- math, science, English writing mechanics, spelling, grammar, punctuation, essay writing -- were a waste of time.  My son is pretty good at math, so it was a big surprise to me to discover that he was not absorbing the online math lessons.  He did not retain the science lessons either.  On the whole, it was a wasted year.

MaryElizabethSings
MaryElizabethSings

@eulb 

I must point out to you that - regarding your son's literature and history assignments online - the retention of facts and information is only the first step of knowledge and understanding.  Answering the higher level questions such as "why" and "what" (not simply the lower level questions such as "who," "when," "where"), and seeing the interrelationships between ideas that manifest themselves in people and events, are the instructional gifts of exceptional teachers who know how to have all students interact within his/her classroom to foster intellectual growth of higher level thinking skills in his/her students, which they will be able to apply to situations in their own lives and in the world's improvement.

Long Memories
Long Memories

I saw that online schools in Georgia and Wisconsin actually did better.  My daughter did the virtual academy from 3rd through 6th grade.  Now a junior in a public school she is at the top of her class in math.  I do believe though it is dependent on the parents following through and having discipline.  Not for everyone for sure but it worked for us.

MaryElizabethSings
MaryElizabethSings

@Long Memories 

Note that your daughter was at the top of her class.  Which came first - the chicken or the egg?  I would think that some very bright students might do well with certain online instruction.
But, we must never assume nor generalize.  The best educators know that and teach that.

Wascatlady
Wascatlady

What a surprise--that on-line education works mostly for the most able and motivated!  So, on-line instruction is not the savior it has been trumped up to be, unless saving money is more important than actual achievement.

MaryElizabethSings
MaryElizabethSings

The condescension and hubris of those in the business arena, toward that which educators accomplish with students, are unproductive.  The Walmart Foundation cannot possibly understand the dynamics of instruction that a trained and experienced educator, such as myself, know already, but who lacks the worldly status and personal wealth of business executives.  Unfortunately, politicians often experience the same hubris toward teachers and instructional leaders, who generally speaking are not political.


If the Walton family wants to make a significant difference in public education, they could take their billions of dollars and employ more specialized tutors to work with teachers to offer support on an individual basis for students who are behind their peers, and for students who need to move beyond their peers.


As I have written so many times on this blog, there will always be a wide range of achievement levels in each grade level for various reasons that I have given.  The way to improve public education is to address all of those various needs.  WalMart Foundation, and Bill Gates, put your money in more tutors in traditional public schools.  The students who are behind need the support and care that a flesh and blood teacher can inspire in them.  Computers cannot do that.


https://maryelizabethsings.wordpress.com/2012/01/15/about-education-essay-1-mastery-learning/

Starik
Starik

Most students won't benefit from computerized learning,  Neither the kids nor the parents have the self-discipline. It's good for kids who can't go to school because of illness. It's good for kids like the kid studying German in addition to her normal schooling. It's not good for kids who are unmotivated to learn.

xxxzzz
xxxzzz

While I have doubts that on-line alone works for most students, this article is sensational and misleading.  It says they are 180 days behind in math.  It doesn't say how far behind they were at the start of the year.  It doesn't say how much of that 180 days they lost during the year. 

MaureenDowney
MaureenDowney moderator

@xxxzzz You can read the actual report on how the lost learning was calculated by following the link that I provided.


But here is a summary of how the comparison between online student performance and traditional student performance was made. This is only excerpt. This link will take you to the full process, which was quite precise and exacting:


http://credo.stanford.edu/pdfs/Online Charter Study Final.pdf


Matched Data CREDO conducted analyses using its Virtual Control Record (VCR) method. The first step in conducting a VCR analysis is to create a matched data set. The matched data set consists of treated students (in this case students attending an online charter school) and demographically identical students in the control group. CREDO established two control groups for this analysis. The first was a traditional control group of students who attend a brick-and-mortar school operated by a traditional school district (brick district).

Due to the dual nature of the treatment group, both online and charter, it was beneficial to make comparisons between the treated students and brick-and-mortar traditional schools and treated students and brick-and-mortar charter schools. This necessitated the creation of a second matched comparison group with students attending brick-charter schools as the control group. This comparison group allowed CREDO to examine the “online-ness” of an online charter school as compared to physical charter schools. 

Selection of Comparison Observations

A fair analysis of the impact of online charter schools requires a comparison group which matches the demographic and academic profile of online charter students to the fullest extent possible. As in previous CREDO studies, this study employed the virtual control record (VCR) method of analysis developed by CREDO. The VCR approach creates a “virtual twin” for each online charter student who is represented in the data. In theory, this virtual twin would differ from the online charter student only in that the student attended an online charter school. The VCR matching protocol has been assessed against other possible study designs and judged to be reliable and valuable by peer reviewers.

1 Using the VCR approach, a “virtual twin” was constructed for each online charter student by drawing on the available records of traditional public school (TPS) students with identical traits and identical. To better isolate the effect of attending an online charter school as opposed to just a charter school, a second VCR data set was created. For the second data set a “virtual twin” was constructed for each online charter student by drawing on the available records of brick-and-mortar charter school students with identical traits and identical or very similar prior test scores who were enrolled in brick-and-mortar charter schools that the charter students would have likely attended if they were not in their online charter school. The second VCR data set using brick-and-mortar charter school students to form the VCRs allowed CREDO to differentiate between the effects of online charter school attendance compared to just charter school attendance. If the effect sizes for online charter students compared to TPS VCRs was found to be similar to the effect sizes for online charter students compared to brick-and-mortar charter VCRs, the effect sizes would be primarily attributable to the online nature of the school. Factors included in the matching criteria were: • Grade level • Gender3 • Race/Ethnicity • Free or Reduced-Price Lunch Eligibility • English Language Learner Status • Special Education Status • Prior test score on state achievement tests

The VCR matching method then eliminates any of the TPS students from the match pool whose demographic characteristics do not match exactly to the individual online charter student. As part of the match process, we also drop from the TPS match pool any students who enrolled in an online charter school in subsequent comparison years. Using the records of TPS students at feeder schools in the year prior to the first year of growth, CREDO randomly selects up to seven TPS students with identical values on the matching variables in Figure 1, including identical or very similar prior test scores. Students with similar test scores were used only when there were not enough TPS students with exact test score matches. The values for the selected TPS students are then averaged to create values for the virtual twin. As all other observable characteristics are identical, the only observable characteristic that differs between the online charter student and their VCR is attendance in an online charter school. The prior test score represents the impact on academic achievement of both the observable and unobservable student characteristics up to the time of the match, the year before the first growth measurement. Since we matched on observable characteristics and the prior test score, we concluded that any differences in the post-test scores are primarily attributable to online charter school attendance. The same process was used for the brick-and-mortar VCR match except feeder list was based on transfers from brick-and-mortar charter schools to online charter schools.

Fulton30350
Fulton30350

@xxxzzz Do they take that into account for 'results' of traditional teachers at brick and mortar schools?

xxxzzz
xxxzzz

@MaureenDowney @xxxzzz Pretty poorly written explanation of the results.  As best as I could tell, the impact of 180 days in math and 72 in reading was over a 5 year period.  They appeared to be looking at 4th to 8th grade.  Spent a lot of time discussing mobility, but I didn't spend the time to figure out how they addressed that in their study.  Most students did online work for only one year ("mobility").  The number who stayed more than two years dropped off dramatically. 

anothercomment
anothercomment

My daughter is currently taking German at the GA virtual academy. Despite landing several major German firms Mercedes, Porsche to name a couple in addition to Siemens which I was the Project Engineer on it Head Quarters bldg in 1984 in Alpharetta., our local schools still ignore the need to teach Gernan.

My daughter had been learning German on Duolingo online and loved it so I merely suggested she take it on the Virttual school and earn credit, credit which would give HOPE boost being a foreign language.

There is a demand for German it was one of the few classes that the single semester AB German I class was completely full. None of the other languages or the AP classes I looked at where. We signed up for Fall semester 16 week session German 1A.

Some of the major problems I have is the audit feature for parents is very very limited only lets you see the grades and if the student has attempted the work. Then they make the student sign a policy that they will not let anyone else use there ID or password and go in as them on their work including parents ( not just other student). I can't from parent mode see or print out the assignments when they are due. Nor do I have access to a text book aka a virtual text book as I would with a real text book so I could go along with my student and help them or review with them. When I ask to, my daughter who sees everything Black and white, hands me a printed copy of the policy and says I can't not let you.

Then my child who I have tried to keep at a rapid pace because I can not see the schedule. She has ADHD . The audit feature was telling me at the beginning of October that she was 83% complete with the course that ends December 4. So surely to God she has all assignments in. Then all of a sudden I get a robo call that her A has dropped to a B. I am like what. I go on line and it says she has two missed assignments one missed completely and one put in the wrong drop box. Then it said she had zeros that we were passed the three day window for resubmitted work. I appealed this all the way up the chain. i told them it was absurd I couldn't see the schedule and class work books through my audit feature. I was told that they found only 4% of parents looked up anything beyond grades in the audit feature so in their last upgrade it was cheaper to just limit parents to that. I told them that was stupid,. You punish the involved and willing to work with their child for the sake that the majority of parents are lazy. I finally had the teacher tell my daughter over the phone she could let me look at the schedule of work due and print me out a copy.

You either go from K-12 were the parent is expected to be the teacher to Ga virtual where they are cutting the children out.

gA virtual needs to do something about the three day late policy when the teachers don't even grade the stuff until after this deadline, so you have no clue if something has been missed. You have to get a notification.

Cobb county has the best parent notification program a parent can set it to let them know of any grades less than 99. That way you know if anything is even one day late.

With ga virtual a parent needs to see the book many. Of us do actually work along with our kids. I have retaken K- Jr. Nursing school with my oldest ( I am not a nurse, but and engineer). I have always been hands on and quizzed my kids, read reports, reviewed them. That is what a parent should do.