Did Georgia — and other states — make mistake turning to homegrown tests?

The state Department of Education released scores today from the 2016 Georgia Milestones tests.

Interesting column by Jim Cowen, interim executive director of the pro Common Core Collaborative for Student Success, on the challenges facing states that chose to create their own tests rather than join the PARCC or Smarter Balanced consortiums.

Georgia is not listed in this inventory of states that had testing problems but it could have been. Georgia pulled out of PARCC, blaming the costs of the new tests. It is not clear Georgia ended up saving much money, and the state experienced problems during last year’s debut of the Georgia Milestones and again this year in the second round. In fact, some high school scores are still not back because all the End of Course tests could not be graded by the promised deadline, leaving thousands of students with “Incompletes” on their report cards.

By Jim Cowen

Last year, for the first time, a majority of states across the country implemented meaningful tests aligned to rigorous education standards. Gone were many assessments that may have been easily subjected to “teaching to the test.” Gone were many of the simplistic exams that simply didn’t provide a true measure of student development. Instead, many states had embraced tests like PARCC and Smarter Balanced which offered higher quality assessments and comparability across participating states.

The impact was immediate. An analysis by Achieve, an independent education advocacy organization, found 26 states significantly closed what we called the “Honesty Gap” — the difference between state-reported proficiency rates and those on the National Assessment of Educational Progress (or NAEP). Unfortunately, some policymakers succumbed to mounting political pressure to scrap the new assessments in favor of independent or state-developed tests.

Those leaders are now quickly learning that “going it alone” is a poor decision. Many thought they could avoid controversy by simply moving away from the PARCC and Smarter Balanced assessments.  However, as Chalkbeat put it, “The process of leaving consortia that was meant to pacify local protests against Common Core-aligned tests has actually led to chaos and confusion in the classroom, not to mention extra costs to those same states to develop replacement exams.

  • Florida withdrew from PARCC in 2013 and awarded a $220 million, six-year contract to the American Institutes for Research to develop and oversee the Florida Standards Assessment. Last March, computer problems disrupted the writing portion of the exam, forcing a postponement. The following month, students again experienced connectivity problems, prompting individuals across the state to request the results not be considered.
  • Indiana exited PARCC in 2013 (lawmakers voted the next year to replace the Common Core) and partnered with CTB McGraw-Hill to update the decades-old ISTEP exam. After coming under fire for technical and scoring problems, lawmakers voted this year to replace the test by the 2017-18 school year. Many experts and educators have expressed concern that does not provide enough time to create a new high-quality exam.
  • Michigan lawmakers voted to not use Smarter Balanced in 2015, just nine months before it was scheduled to be administered. It used the M-STEP that year, followed by a different M-STEP exam in 2016, subjecting students to three different tests in as many years. Some in state government are already considering changing the tests again next year.
  • Mississippi withdrew from PARCC in 2015 and later signed a $122 million, 10-year contract with Questar Assessment Inc. This spring, 12,000 students across the state experienced connectivity problems on the electronic exams.
  • Tennessee exited PARCC in 2014 and contracted with Measurement Inc. to administer the TNReady assessment. After months of technical problems, which forced the state to revert to pencil-and-paper tests—which many schools never received—Tennessee terminated its testing contract. Officials partnered with Pearson to grade the tests that were completed this year. The state paid $1.6 million of a $108 million contract with Measurement Inc.

The list goes on and on. Even states that never adopted the Common Core or assessments aligned to the standards have experienced testing problems.

  • Alaska cancelled the computer-based Alaska Measures of Progress this year after a fiber optic cable was severed, which caused the testing platform to crash. The test cost the state $5 million annually, and officials do not know whether the state will be reimbursed.
  • Texas—which also never adopted the Common Core or associated assessments—experienced a different type of testing disruption. Parents sued the Texas Education Agency to block officials from using the results of the STAAR exam. The scores are invalid, the lawsuit alleges, because the tests were not designed to be completed by a majority of elementary and middle-school students within two hours (for grades 3-5) or three hours (for grades 6-8).

Beyond the costs, time constraints and technical challenges that accompany the development and implementation of new assessments, states that have struck out on their own have also jeopardized their ability to compare their progress to other states—and may very well come out with an inferior assessment in the process.

One of the chief benefits of comparable, high-quality assessments is the ability for education officials, teachers and parents to compare how well their schools are preparing students relative to others across the country. That information allows families to hold local authorities responsible and to ensure their kids receive a quality education. It allows teachers to gauge their performance, and, more importantly, to collaborate with their peers across district and state lines to build on best practices. By going it alone, state leaders largely forfeit that capacity. This shortsighted, short-term political solution, ignores the economic reality that students today are not simply competing with those in the desk next to them, but also with students on the opposite coast and across the oceans.

What’s more, there is little evidence to suggest whether states’ new exams are high-quality, or if they mark a return to the poor content state leaders worked for years to replace. By contrast, mounting evidence reaffirms the value of both consortia exams. Across the country, states have raised classroom expectations and begun to measure progress toward those higher standards with high-quality, comparable assessments.

Leaders should resist temptations to go it alone, otherwise they risk undoing the years of work to get to this point. High-quality student assessments are one of the strongest tools teachers and parents have to ensure students receive the support they need. That shouldn’t be surrendered to the political winds of the moment.

 

Reader Comments 0

25 comments
eulb
eulb

I appreciate all the links embedded in the article.  They are helpful.  The "Honesty Gap" link leads to an interactive USA map where one can compare NAEP results on a state-by-state basis.  I found GA, SC, TX, VA, and MI to be very interesting. 

Michael2255
Michael2255

I will never understand how the most stupid among us wrest control of education from the educated.  Ahh, democracy, where are school boards are littered with elected high school grads, bible thumpers and real estate agents.

CSpinks
CSpinks

Georgia did. Don't know enough about the circumstances in other states to have informed opinions about their situations.

Another comment
Another comment

The public schools need to have the same testing that Westminster, Lovett, Martist, etc have.

WhoReallyCaresWhoIAm
WhoReallyCaresWhoIAm

@Another comment Those schools can be-and ARE-selective about who they select to attend.


Not so with public schools, which take everyone...the results wouldn't even be close, and you're pretty much comparing "apples to oranges."

Beach Bound2020
Beach Bound2020

I often wonder why we would even waste money on additional tests when for the most part we are already using great reliable, valid, nationally normed tests for our children.  Merely making little adjustments to frequency and scope of administration, there is no reason not to rely on the ITBS, Cogat, ACT, Readistep, PSAT and SAT to monitor learning, growth and progress for our students.  The metrics are already created. The data easily comprehensible by parents and students and the cost minimal compared to what's being paid now.

dg417s
dg417s

@Beach Bound2020 The only problem with the tests that you mention is that they are designed to measure individual students against a bell curve, so half will be below average. That is the way it is supposed to be. Georgia Milestones, in theory, are supposed to measure student understanding of the standards devised by the state. That being said, there really is no way to say for sure that is actually the case. I can't see the test to know if the vocabulary that I am teaching my students is the same as what is being printed. I can't verify that the questions are valid. I can't ensure that the questions match the standard. I can say that these outside companies don't always do a good job for that. Several years ago, my district purchased tests from Pearson and I was part of a team that reviewed the questions. We rejected 40% for being too vague or not aligned to Georgia standards. How do we know that isn't the case again with the current company? Like the Tootsie Pop commercial says: "The world may never know." The simple solution - allow Georgia educators to design the tests and remember that the tests are designed to test student understanding of material and nothing else. I would even recommend doing the tests as a benchmark series rather than a cumulative EOC or EOG. It would make the tests much less daunting on the teachers and students (here's a quick 15-20 question benchmark over standards 1-3 for example) and give those several times a year and move on. That would give teachers the immediate access to data that they need to improve instruction. A bunch of circles on a page for students who walked across the stage last week does me no good.

WhoReallyCaresWhoIAm
WhoReallyCaresWhoIAm

@Beach Bound2020 You are mistaken if you think CogAT (mental ability tests, primarily used for gifted identification in publics) and ITBS are used widely. Most districts stopped because of the price tag.


GAMAS is a joke-in scope and implementation. 


NWEA's Measures of Academic Progress-(MAP)-given 3 times a year-would be a great alternative to the ITBS. 

LaKeisha
LaKeisha

@dg417s @Beach Bound2020 I'm not sure I agree with you here, dg.  The ITBS, for example, is exactly what its name claims to be:  A test of basic skills.  It's possible for everyone in a given classroom to make a very high score...if that is a fair representation of their ability.  It is also possible for everyone in a given classroom to fail it dismally...if THAT is a fair representation of their ability.  It is certainly true that if you take all the ITBS score results nationwide, they will probably reflect a bell curve...because that is a fair representation of the intelligence level of a given age group.
Your point that an exam should test student understanding of the material taught during the year is valid, provided that the material taught reflects an accurate grouping of the appropriate material that students SHOULD be taught during a given school year.  

Starik
Starik

Half the states are below average in their educational performance, Georgia included.  They spend taxpayer money on tests they can manipulate to conceal the facts: they have bad schools, below average students or both.  We have a national government.  Let it handle the testing.

FlaTony
FlaTony

@Starik By definition, half of a sample must be below average and half must be above average. That's how the math works.

Wascatlady
Wascatlady

The reasons states have shied away from the PARCC tests seems to be (1) apparent cost, (2) they are "tainted" by our present President's involvement, no matter how incidental, and (3) most importantly, states cannot "adjust" the cut scores to make themselves look better, or show "growth."


I don't know if those national level tests are any worse than the Georgia state-derived tests, but I know many "leaders" who do NOT want to give up any control over scores and spinning the scores.  By my observation, THAT CONTROL is key.


So the teachers, students, and parents are left to deal with the "results." and the taxpayers get close to nothing for their money.


The question is, who profits, both financially and politically.

FlaTony
FlaTony

A very biased essay from the outset. The concluding remark shows how distorted the views on assessment have become. The so-called "high quality" assessments DO NOT provide useful information for teachers to use when making day to day decisions about students' needs for learning. That is a fallacy in their arguments. What is real is the fact that the testing business wants the big money that comes to them when all students are required to take their tests.


A second and very disturbing aspect of modern day assessment is that it has been linked to the notion of accountability. Assessment should be about student learning and making determinations of their progress toward goals that are set by schools, districts, and states. Any other use of student assessment goes against the fundamental purpose of the test.


The money that has been diverted away from students and into the pockets of the testing companies would be better spent making sure all students had access to better teachers, supplies, and meaningful resources for the classrooms.

trifecta_
trifecta_

No test will ever be acceptable to the anti-accountability crowd this "education" blog fronts for: mainly teachers' union types. 

And no exaggeration too ludicrous.

They find value in tests which serve their ends, but none whatsoever in those which allow parents to compare outcomes.

redweather
redweather

@trifecta_ And people like you will only be happy once all schools are charters with little accountability.

CMK3
CMK3

As a parent and educator, I haven't found anything beneficial on the score reports! The main focus of the results page seems to be a comparison of a child's score with average scores from the state and local populations. Even the 20 question national normed percentile doesn't always"match" the CR score. Too much time testing for comparison reports instead of information that can be used to enhance learning.

trifecta_
trifecta_

@redweather 

I'll be happy when parents are free to apply their child's tuition voucher to their school of choice. Public or private.

And you're free to compete for their business.

WhoReallyCaresWhoIAm
WhoReallyCaresWhoIAm

@trifecta_ @redweather Do you think that the voucher that equals GA's contribution will pay for a good private school education? Check out tuition rates...not even CLOSE.


And stop already with the "teacher unions/teacher union types." I find it interesting that the states with strong unions are KILLING us.


#Pocahontas
#Pocahontas

@redweather @trifecta_ 

Catholic schools cost far less than the $14,500 spent yearly per student by Atlanta public schools.

And Chicago, Detroit, Los Angeles and Washington DC all have strong unions but terrible results.

Wascatlady
Wascatlady

@#Pocahontas @redweather @trifecta_ If Catholic schools (and I have nothing against them) had to take ALL who come, including the most handicapped, the worst-behaved, AND HAD TO KEEP THEM, their costs would be much higher also.

dg417s
dg417s

@#Pocahontas @redweather @trifecta_ Detroit is under the control of an emergency manager (think OSD superintendent) and they have mushrooms growing out of the walls, buildings falling apart, etc. Washington DC was under the control of Michelle Rhee and saw cheating just like under Beverly Hall. Chicago is run through the mayor's office, and he's President Obama's friend (think RTTT). If you're going to cherry pick the strong union districts, make sure you provide all the facts.

#Pocahontas
#Pocahontas

@dg417s

Your point? None of those unionized school districts have had good results in our lifetime.