Dear CTB/McGraw-Hill, Give back our money for problem-prone Georgia Milestones tests

A University of Georgia math education graduate student and middle school math teacher, Zachary Kroll writes a letter of complaint to CTB/McGraw-Hill on the inaugural run of the Georgia Milestones tests.

CTB/McGraw Hill has a five-year,  $107.8 million contract with the state to create the new standardized test. Kroll contends the company owes Georgia a refund for failing to deliver an effective and efficient testing product this year.

By Zachary Kroll

BarryMaguire.NewsArtThe purpose of standardized testing is to ensure everyone within the public education system of a county, state, or country is taught to the same level of rigor. Unfortunately, we have moved so far away from this idea and are now testing for the sake of testing.

Teachers are being forced to move away from teaching life skills, such as critical thinking and problem solving. They are now required to shoot through as much material as they possibly can within an absurdly short period of time. This has changed the school environment from a friendly, exciting, and caring one to a more stressful, anxious and less happy place for people to come to each day.

I never remember my teachers complaining or freaking out about making sure we covered every piece of material that was going to be on the end of grade/course examination. It is important to note I grew up in a community with excellent schools. This does not change the fact the testing culture has gotten worse over time.

Your company was awarded the contract to design and administer the Georgia Milestones exams. The Georgia Department of Education awarded CTB/McGraw-Hill a five-year contract worth $107.8 million. With this came the expectation we would receive the most effective and efficient testing programs available today.

On your website, you promise by 2019 all of Georgia’s testing will be done online. This is a wonderful goal; as long as you can pledge it is going to work for every student taking these tests.

During this school year, we wasted numerous class periods testing the system, making sure our students can log in, and checking to make sure your technology will work when we need it most. This has taken valuable days away from our teachers, administration, media specialists and, most importantly, our students. These days could have been spent learning.

We, as educators, try to instill in our students a sense of pride, self-efficacy, and responsibility. It is our hope when our students move on from our classrooms they feel better about themselves and trust the work they did was their very best.

I want your company, especially those in charge, to question whether you have done your absolute best when you designed this program and this software. Did your designers and employees put forth their best effort? Did they make sure there was nothing else that could be done to guarantee our teachers and students would not have any issues on testing days?

Based on my experience over the course of the last several months and especially the two weeks of testing, the answer is a resounding NO!

If it were yes, then test administrators, hallway monitors and other school faculty would not have been running around all morning fixing computers that were unable to get into the test. We would not have spent 30 or more minutes of the test period dealing with a system that would not load, a program that froze, or a computer that was working fine until a student logged onto the system. If the testing went as you promised, we would not have had students with accommodations being forced to restart their exams multiple times.

Testing is a stressful time for every party involved, and it is your responsibility to make certain the one thing that does not add to that stress is a software or program error. CTB/McGraw-Hill did not do its job.

Because of this, I ask that you renegotiate the contract you signed with the Georgia DOE and return $21,560,000 to the state. This is one-fifth of the amount of the contract, and I think it is fair to say that you did not earn it this year.

The wonderful thing about this, though, is there is always next year. I encourage you all to take the summer and ask yourselves the questions I posed. Really determine if this software is best for our students. If it is not, then find a way to make it better.  At the end of the day, our No. 1 priority is to give our students the best opportunities to learn.

 

 

 

Reader Comments 0

49 comments
Carlos_Castillo
Carlos_Castillo

Keep in mind that McGraw-Hill is the best of the lot.

BCW1
BCW1

They seemed to have underestimated the volume of test taking that was going to take place. And for it to happen with the screen reader accommodation groups was not good either. Those groups struggle with testing when everything goes well as it is.

popacorn
popacorn

Note to teachers: Make sure all extension cords, computer cords etc are plugged in. Power buttons should be in the 'ON' position. 

MD3
MD3

@popacorn Yes, because I'm sure that's where the bulk of the problems originated...

jerryeads
jerryeads

And just happened to run into an Edweek editorial by Marc Tucker, who has been a major force in the standards movement for decades. His closing paragraphs from "How should we gauge student success?":

"The United States has responded to poor student performance by instituting draconian high-stakes accountability systems that create very strong incentives for teachers to teach only a small portion of what they should be teaching and, indeed, want to teach.  The great irony here is that, since these high-stakes accountability systems were introduced, there has been no improvement in student performance at the high school level in the things the high-stakes tests measure, while, at the same time, there is every reason to believe that our students are doing far worse on the important things we should be measuring but are not measuring. 

"That is a terrible deal for our children and our country."

Finan
Finan

@jerryeads 

So give parents the right to choose the schools, and those who believe as you apparently do can send their kids to unreformed schools.

Astropig
Astropig

@Finan @jerryeads


"So give parents the right to choose the schools, and those who believe as you apparently do can send their kids to unreformed schools. "


Or, conversely, come up with a better system of gauging student success,copyright it,sell it to school systems and there's a great opportunity to do well by doing good.In the business world (ooooooh! dirty words!) , groups of employees often get together to found new companies that take a different approach to a problem than the industry norm.Those people sometimes end up fabulously wealthy and the world has an...Apple computer or a similar innovation that makes everybody's life better.You don't have to get wealthy- Dr. Jonas Salk,I'm looking at you- but you can improve things rather than just complaining about what does exist.


This is the one thing that I haven't seen from the people closest to the testing/accountability complex. I haven't seen any teacher (or group of teachers or administrators) design a better system for measuring student improvement.They're apparently waiting for a "top down" approach from just the right administration in D.C. or some other miracle like parents and political leaders just doing what they'd like spontaneously.

smithmc
smithmc

@Astropig @Finan @jerryeads I can just hear it--teachers spend their "free" time creating tests to replace the flawed corporate ones--and you'll hear cries of how teachers should be TEACHING, not writing tests!  


Whatever teachers do, they can't win, because the corporate-media complex have invented a "problem" that doesn't exist, and now you're asking them to fix the mess too.

jerryeads
jerryeads

Finan - spending money on tests that don't work doesn't make better schools. We've been doing this sort of testing for fifty years to absolutely no avail.. Spending that money on better facilities and resources (including better teachers and school leadership) for the "failing schools" that mostly "suffer" from disadvantaged kids from poor and broken (or no) homes might.

Teacherandmom is right. This is relatively recent technology that requires sufficient infrastructure to work. One of these days it will. But if all we're doing is multiple-guess testing using a screen instead of a scan sheet, see first paragraph.

Finan
Finan

Dear school districts: Give back our taxpayer money for those failing schools which make achievement testing an absolute necessity.

living-in-outdated-ed
living-in-outdated-ed

I have absolutely no problem with what the author penned about the test.  i just wish that educators would edit their writing for typos and grammatical errors!


I am all for online testing, but there can be absolutely NO SOFTWARE GLITCHES. No one can argue that a student is in a very stressful situation when taking a standardized test.   If there were glitches, then we have added to such stress and this can have quite an adverse impact on their performance.    And I'd love to know how much learning time was lost by testing and implementing the new test protocol.


As such, there should be some level of refund for what has occurred.  I don't think a full one-fifth is realistic, but it should be a very meaningful amount.  


But see - Barge dropped PARCC in part because he thought the price was too high.   So you get what you paid for.

TicTacs
TicTacs

I wouldn't hold my breath...


MaureenDowney
MaureenDowney moderator

Some of these comments miss the big picture questions here.

From the start, there were concerns whether Georgia was spending enough to get tests that went beyond A, B, C, or none of the above. Good tests are expensive, which is why Georgia pulled out the multi-state consortium seeking to create tests of higher quality. (There are questions about those new consortium tests, as well. Namely, are the tests too complex and are the questions relevant and clear?

For me, here is what we need to know:

Did Georgia get what it paid for? (Or, in this case, perhaps didn't pay for?)

Is the problem a lack of technology infrastructure necessary for a successful transition to online testing?

Did the vendor work closely with DOE to ensure tests aligned with curriculum, especially given the changing of the guard at DOE?

Are the reported snafus coming in from all corners of the state part of a normal roll-out or a symptom of serious flaws?

bu2
bu2

@MaureenDowney 



"Did Georgia get what it paid for? (Or, in this case, perhaps didn't pay for?)"  That is the relevant question.  Does the test really measure what it is supposed to.  I haven't heard any discussion of that, just the technical issues.



My child didn't report any problems.  Now they did a dry run a couple months ago and had lots of issues.  I suspect these technical snafus are due to insufficient preparations by the local school, not the vendor.  It sounds like a lot of the anti-testing complainers grabbing at anything they can when the technical problems may well be due to incompetence at the school level.

OriginalProf
OriginalProf

@MaureenDowney 

My own "big picture question" is why the state keeps issuing educational commands that necessitate all state schools to be able to use computers?  Clearly, not all state schools will be able to comply.

Batgirl2
Batgirl2

@bu2 @MaureenDowney You probably won't  hear if the test measures what it is supposed to because teachers have been threatened with the loss of their certification if they discuss the test.

bu2
bu2

@MD3 @bu2 @MaureenDowney 


Then why did only some schools have problems this time around? That sound like a local issue.  Otherwise it would have been like the trial run in February.  Your bias against testing is the only thing that is evident.  As for the problems with school IT, all you have to do is listen to "educators" complain.

MD3
MD3

@bu2 @MD3 @MaureenDowney Again, your bias is apparent, right down to the fact that you put the word educators in quotation marks. And, "only some schools" didn't have problems this time around. Most schools around the state whose special-ed students were using the screen-reader application experienced significant problems, to the point where the state finally had to advise schools all over the state to stop trying to test those kids, and then apologized pretty profusely because they knew what a disaster it was. As for your other misguided contention that I somehow have a "bias against testing", nothing is further from the truth. I want quality assessments that give me useful information about my son's academic progress. The CRCT never did that, and there's no indication that the new assessments will either. We have spent hundreds of millions of taxpayer dollars to testing vendors, and the return on that investment has been pathetically insufficient. 

One more note... if there are problems with IT infrastructure in schools that are leading to complaints, I would bet that a significant portion could be attributed to the $8 Billion that the state has shortchanged the local school districts over the past decade. It's kind of hard to maintain a top-notch network when you face those types of shortfalls. 

But again, don't let facts get in the way of your ideology...

teacherandmom
teacherandmom

@bu2 @MaureenDowney Wrong bu2:  The issues were not at the system level.  The problems were with CTB and their servers.  Particularly the screen readers.  The DOE requires EOG assessments be completed within a 9 day testing window. Many systems chose to test toward the end of the testing window (March 31-May 1) to allow for more instrucational days.  This meant the CTB servers experienced a peak in traffic last week.  If your child completed his/her assessment before last week, they probably did not experience any technical issues.  


We also had random glitches that included students logging in and the computer skipped  part 1, section 1 in math and placed them in part 1, section 2, then returned them to the first part. We had issues with students attempting to review their test answers only to have them kicked off the platform and their test submitted for grading.  Due to the lag time with the CTB servers, some images would not load and the students had to wait to click on reading passage tabs.  ALL issues with CTB.


I'm glad your child did not report any problems.  I resent the accusation that my district did not adequately prepare and by pointing out the glitches in the system earns me the "anti-testing complainer" label.  


I actually like online assessments.  Identifying the glitches is the first step to improvement.  





teacherandmom
teacherandmom

@bu2 @MD3 @MaureenDowney See my reply to your earlier comment.  If schools tested earlier in the testing window, they did not experience as many issues because the traffic was limited.  The problems started when the trafffic picked up toward the end of the window.  More systems went online and BAM  the servers slowed down.


Here's an analogy for you....If my commute into Atlanta was 3am every morning, would I be correct in assumming Atlanta does not have a congested traffic problem....

bu2
bu2

@teacherandmom @bu2 @MaureenDowney 

Unlike yours, most of the complaints seem kind of whiny and generally attack testing and capitalism, etc.

Yours is a precise, logical response which is something that gets taken seriously.


If some schools have problems and some don't when they take at the same time, the more logical assumption is that it is a local problem.  Some on here have commented that it went smoothly at their school as it did at my child's where it was done the last 2 weeks.  There are lots of places along the way where things can go wrong, including the school's own internet connection.



bu2
bu2

@MD3 @bu2 @MaureenDowney 


There's no ideology.  Only facts.  Only certain schools had problems.  My child tested in the last two weeks like most did.


Your tone is the reason many critics don't get taken seriously.  You accuse me of bias because I don't buy your explanation which seems inconsistent with actual results.  Then you get upset over quotes reading something into that isn't there.  Emotions are overrunning logic.  The quotes are because of your imprecise term.  Educator can mean a teacher or a superintendent or principal or anyone else in the system.  If there is a local problem, that's the school's IT and testing administrators, not the teachers.

MD3
MD3

@bu2 @MD3 @MaureenDowney But it wasn't a "local problem". It was state-wide, and as I've already said, the state admitted that there were problems state-wide, and issued apologies for those state-wide problems. If the kids at your local school didn't have any issues, then great for them. But that wasn't the experience for most of the schools in the state. 

bu2
bu2

@MD3 @bu2 @MaureenDowney 


No, what you said above was that it was a statewide problem during the trial runs and the state apologized then.


MD3
MD3

@bu2 @MD3 @MaureenDowney Wrong. Try again. I said that during the readiness test, the CTB server failed spectacularly, which it did. No apologies from the state then. They just said that was valuable because it gave them information which would help with the actual test administration. THEN, when the bulk of the state started testing last week, the screen reader application froze state-wide because the servers were overloaded, couldn't handle it, and districts across the state had kids trying to test on an unresponsive system. That was when the state apologized - because it was the actual administration of the test and they knew what a disaster it was. 

Again, if no one at your kid's school had issues, then great for them. They were lucky. But that was by no means the norm.

teacherandmom
teacherandmom

@popacorn @teacherandmom Idiot...of course I did.  I can also tell you exactly how much  bandwidth our district was using during online testing...which was actually below the amount we use on a regular school day.  I can also assure you the screen readers stopped working throughout many districts at around the same time because we were communicating not only with the schools in our district but neighboring districts as well.  


Our first course of action was to bring another wireless router into a classroom to see if that improved the screen readers.  It did not.  By that time other districts were reporting the same issue and we realized the problem was not local.  The CTB servers were the problem.


Within hours the DOE confirmed the problem was on CTB's end. The DOE's support has been exceptional.  


End-of-Grade testing concluded yesterday.  We start EOCs  next week.  We are hoping for a smoother online testing window but have a Plan B and Plan C in case the screen reader issue happens again.  


I know how to do my job and I do it well...but thank you for the unnecessary reminder to always check the power cord :)

MD3
MD3

@bu2 @MaureenDowney  "Now they did a dry run a couple months ago and had lots of issues.  I suspect these technical snafus are due to insufficient preparations by the local school, not the vendor."


You would be wrong in that suspicion. Schools around the state participated in the "readiness test" and the CTB server failed spectacularly. Most local school systems had no problems on their end with bandwidth or network services. The server could not handle the traffic. Likewise, the snafus with the screen-reader application that is a special-ed testing accommodation was also caused by the CTB server, and not the local networks. The state has admitted that and apologized for it.But go ahead and keep assuming that it's just a bunch of teachers complaining over nothing. Your bias against our educators is very apparent.

straker
straker

It seems this is just the latest social experiment designed to make test scores equal for all students, Black, White, Asian and others.


As usual, it has not and will not work.


But don't worry!!


Your may be certain that yet another experiment in equal test scores for all is waiting in the wings.

class80olddog
class80olddog

The problem with every type of test used is:  THEY DON'T USE THE RESULTS!


What good is it to test a student and find out they are reading three years behind grade level THEN PROMOTE THEM TO THE NEXT GRADE ANYWAY!



class80olddog
class80olddog

@Wascatlady @class80olddog  So what good is a test that you don't use the results for the student?


Just another excuse not to address key issues (social promotion).


By the way, when is the AJC going to do an expose on cheating on attendance data?

PJ25
PJ25

Just give the teacher's big raises and don't hold them to any state or federal standard.  This is the only way they will ever be happy. 

MiltonMan
MiltonMan

"If it were yes, then test administrators, hallway monitors and other school faculty would not have been running around all morning fixing computers that were unable to get into the test. We would not spent 30 or more minutes of the test period dealing with a system that would not load, a program that froze, or a computer that was working fine until a student logged onto the system. If the testing went as you promised, we would not have had students with accommodations being forced to restart their exams multiple times."


Is this clown for real????  I have visited numerous schools that, at best, had antiquated "PCs" that were not suitable for anything much more than playing solitaire - which numerous students I spotted were playing.  When I questioned the schools' "IT Coordinator/Administrator", the typical response was "we use what we are given".  I also found numerous unopened software CDs just laying around and uninstalled software updates, patches & notices - a simple task for the coordinator to conduct.  We often would leave the schools with a "Recommendation List" to which many did not implement when we returned later on in the school year.  I quit doing this "volunteer service" due to lack of responsive enthusiasm from the schools.


This grad student is missing the point but will make a fine addition to the education community by blaming others for his incompetency.

sneakpeakintoeducation
sneakpeakintoeducation

@MiltonMan


The school I worked at last year had a computer lab that was so out of date and by the time all the children logged on the computer lab session was all but over. 

4PublicEducation
4PublicEducation

AMEN!  Headline should be  CTB/McGraw-Hill FAILS GEORGIA MILESTONE TEST.

AJCkrtk
AJCkrtk

As was noted by someone on FB:  "Is the computer hardware sufficient to run the software? How about school systems without newer technology? Has an IT analysis of each school systems capabilities been conducted prior to piloting online tests? I have been in many work situations where the software was fine, but there was no money to upgrade the hardware, and the software froze, ran too slowly or would not run at all."  I agree.


At the very least,  CTB/McGraw-Hill definitely needs to reimburse that portion that would cover APS - APS is not even able to use computers for the test this year.  From the APS website:


Note: The Georgia Department of Education has informed Atlanta Public Schools that the vendor for our Georgia Milestones Assessment System online component (CTB McGraw-Hill), updated their testing client (CTV) for the assessment on January 8, 2015. This update is not compatible with our district’s current technology infrastructure (Citrix VDI environment) for the 2014-2015 school year. The State is continuing to work on upgrades and a solution to this compatibility issue.

The Georgia Milestones End of Grade and End of Course assessments will be administered using paper and pencil for all of our APS schools, including our charter schools. Atlanta Public Schools will continue to administer our state assessments with the highest security possible. Atlanta Public Schools is working with Georgia Department of Education to move towards online administrations in future years. 

redweather
redweather

@AJCkrtk That says it all.  But for some posters here, it's still APS's fault.

edawg
edawg

Having watched my students' test freeze then they had to finish with a paper test ... yes!  I totally concur.  If teachers are expected to be measured by testing data then the state & CTB should be measured by how well their online system worked.

Wascatlady
Wascatlady

@edawg They get a big old "Does not meet expectations" on this one.

JaneInCobb
JaneInCobb

Sorry, but I'm guessing no test will ever satisfy those who want federal education dollars with absolutely no strings (or expectations) attached.

Quidocetdiscit
Quidocetdiscit

@JaneInCobb


So suggesting a test for which the state paid millions of dollars should run smoothly without the numerous glitches that many of us giving the test had to deal with is somehow akin to wanting "absolutely no strings (or expectations) attached"?  


Hyperbole much?

popacorn
popacorn

'At the end of the day, our No. 1 priority is to give our students the best opportunities to learn.'

Huh? This is about testing. Does every educator have to reflexively regurgitate this same line over and over about any topic whatsoever? The computer problems will be worked out. No computer program/software/platform has ever worked perfectly first time out. Trust me, your students pick up on your frantic overreaction, educators. The sky's not falling. 

dg417s
dg417s

Considering that as of last month, we don't have a waiver for test scores being 50% of our worth as educators but students aren't held accountable for their scores, yes, you could say the sky might just be falling.