Damned by data: Do states spend more time rating schools than helping them?

States collect piles of data to rate their schools but are the ratings useful? (AJC File)

In this age of data worship, state education agencies seem to devote more time to rating schools than helping them. Many state legislatures have embraced a “damned by data” strategy to force change in public education. As a result, school districts are neck-deep in data collection mandates.

In response to a state law passed by the ever-helpful General Assembly, Georgia has introduced another category of school ratings, the Financial Efficiency Star Rating, released last week. My problem is most new ratings don’t help anyone, schools, students or parents.

Some ratings, such as Georgia’s College and Career Ready Performance Index, are unwieldy and overly complex. For example, points awarded for student growth can mask low performance. Yes, students are improving but they may be a long way from proficient, which parents can only know if they look behind the final CCRPI score to all the contributing categories and bonus points.

On the other hand, the state’s new financial efficiency rating is overly simplistic, yoking a three-year average of per-pupil spending and CCRPI scores to award a star rating of one to five, with one being the worst and five the best.

I’ve already expressed my skepticism about the state’s school climate ratings.  Allow me now to throw some shade on these new financial efficiency ratings.

The academic challenges students bring to the classroom affect spending. More challenges require more resources. Plus, location influences salaries; the metro area has a higher cost of living than south Georgia so pay scales are higher.

It should be no surprise Atlanta Public Schools – with deep multigenerational poverty to overcome — spends more per student than Forsyth, which has become a county of choice for middle-class transplants to Atlanta.

Gwinnett, the state’s largest system, earned a high score, in part because of the low-per-student cost derived from the district’s economies of scale. Gwinnett has been able to raise student performance with far larger-than-average schools. City Schools of Decatur, on the other hand, earned a low score; it has deliberately maintained smaller schools so it doesn’t see any economy of scale benefit.

These financial efficiency ratings represent a rough-hewn measure that rests on the validity of CCRPI, which rests almost entirely on the validity of the state Milestones tests. And that is an unproven foundation since the tests are so new.

The state Department of Education issued its own caveat:

While the star rating provides additional information about the relationship between Per Pupil Expenditure and student achievement, users should note its limitations. Although all districts follow the same reporting guidelines, inconsistencies in the use of school codes have resulted in the districts reporting similar expenditures in different ways to the GaDOE. For example, a school district that reports teacher salary expenditures at the district level will result in very high district allocations relative to a district that reports teacher salary expenditures at the school level. The Financial Review Division has proposed clarifications on the use of school codes, and the revision to the Handbook is expected to be released in March 2016. This rating is designed to be used in conjunction with other information to obtain a holistic assessment of a school or district.

According to AJC education reporter Ty Tagami:

In core metro Atlanta, the Gwinnett County Public Schools achieved the best score, 4.5, and Atlanta Public Schools the worst, 1.5. Forsyth County, beyond the Perimeter but still in metro Atlanta, had the best score in the state, scoring the only 5 out of all 180 districts. Forsyth scored well for academics but also was among the 10 districts with the lowest cost per pupil, at $7,115 averaged over three years. Atlanta, however, had the second highest cost in the state, at $13,297 per student.

Fulton County Schools fared nearly as badly as Atlanta while DeKalb County Schools and Decatur City Schools were below average. The Cobb County School District did alright, but Marietta scored poorly.

You can check out your system’s rating here. You have to choose a school district and then  select a report for elementary, middle or high school. Click on the “Financial Efficiency” tab to find the overall district rating, which is displayed in each of the three report types.

Districts could respond to the scores and explain the causes of their results. You will see the responses on the state site. However, only a few districts chose to provide an explanation. One was APS, which said:

APS faces unique challenges impacting this rating.

(1) Atlanta has one of the highest costs of living in the state, which has an impact on salaries needed to retain quality employees.

(2) APS has an unfunded pension liability with an increasing obligation until 2027.

(3) APS students require additional resources with a large proportion of students in high-need and high-cost categories such as special education, ESOL and poverty.

(4) Generous partners are investing in schools and increasing per pupil expenditures, but not using taxpayer dollars.

(5) APS maintains low-population neighborhood schools, which may yield greater per pupil expenditures. Expenditures in APS have been and continue to be driven by efforts to increase student achievement. In 2017, APS is investing $24M in a turnaround strategy for low-performing schools. APS is committed to making students career and college ready, and expects to continue significant investments in order to improve student achievement.

 

Reader Comments 0

21 comments
MaureenDowney
MaureenDowney moderator

I wanted to share a note from a research scientist. He wrote: 


 These indices are becoming increasingly problematic and way for some schools especially large districts to obscure and spin data. From a pure statistical standpoint reporting school level data is a problem because:


1. Enrolment figures are very unstable and some districts have considerable migration almost on a daily basis. It’s not unusual to see kids move 3+ times within a semester. Radical changes or inaccurate estimates with respect to the denominator results in misleading data;

2. Principals have direct control over reporting and can easily underreport/over report data as they see fit and it is very difficult to detect;

3. Almost all of these indices are based against Title 1 schools

class80olddog
class80olddog

You are spot on the money with your comment that a higher CCRPI score can mask low performance with its built-in grade inflation for "improvement" and "poverty".  Look at Conley Hills Elementary School.

class80olddog
class80olddog

The point that is missed about this Financial Efficiency Star Rating is the premise that more money does not give better results.  You say "but more money is needed for needier students".  More money does not even bring them up to AVERAGE. That is because the money goes towards things that do not help pull up the failing schools.  No money for attendance, discipline, or combatting social promotion.  Just more money to feed students whose parents should have fed them. More money for "feel-good" policies and educational doublespeak.  More money for PC issues.  More money to try to turn a 60-IQ child into the next Einstein. 

elementary-pal
elementary-pal

@class80olddog  Or maybe more money for more teachers.  More money for after-school tutoring programs.   More money for supplemental instructional materials.  More money for after-school buses so students can stay for extra help.  More money for field trips so students can have real life experiences with science, math, and social studies.  More money to pay teachers for an extended school year.    

Itsbrokeletsfixit
Itsbrokeletsfixit

@elementary-pal @class80olddog More money is needed, yes. But the real problem is that MONEY is the driving force in structuring education, when we should be looking at education in a different way. What does that mean? 

Our childern are the greatest asset our nation has. And providing them the best possible education is our greatest hope for the future. We know a lot about how to educate but, instead of applying what we know to build the best education we can for our children, we continue to compromise basic principles and substitute weak alternatives in the name of saving money. Thus class sizes are too large, teachers are not as well trained as they should be, support staff like counselors, nurses, are to few or missing altogether, support for academic competetions doesn't exist (unlike support for sports competition), and media support for academic success is almost non-existent. We can build multimillion dollar sports arena's, and we can continue to spend trillions of dollars for the best military in the world (we spend so much on the military that we can't even keep track of the money!) but we are unwilling to look at public education the same way we view our military support. Namely we should:

1) Clearly state the objectives of a K-12 education, defining what key elements a child should know AND BE ABLE TO DO when he graduates from HS. 

2) Using educational research from the past 100+ years, define the strategies needed to educate our children so that they can achieve those objectives. 

3) Then, we find ways to pay for top quality education, and get rid of all the "reformist" snake oil solutions being forced upon public education for the past 15-20 years. 

The military has been able to frighten our nation into providing whatever funding they want for at least 150 years while we have never funded education adequately. 

Now that the problems of educating our most vunerable children (about 1/4+ of them) have become evident, instead of trying to get to the real problems, our government(s) are blaming the underpaid, over-worked school teachers who are trying to cope with impossible situations. 

If we do not wake up to the impending disaster that is being thrust upon us with standardizing testing, charter schools, common core, RTTT, NCLB, and the rest of it, our support for the military won't matter because our nation will rot out from the inside.  

Karen Davis Rose-Jackson
Karen Davis Rose-Jackson

I like APS's use of the word, "invest" and "investment." Students should not be seen as expenditures.

Astropig
Astropig

Why is the collating and disclosure of this data such a bad thing? Doesn't the public have a right to know as much as possible about their schools?

MaureenDowney
MaureenDowney moderator

@Astropig One of the issues is the time and staff required to collect data that no one confirms -- not enough staff at DOE to do so. Not enough funding to hire a staff.

In 2010, the feds audited Georgia and found undercounts of dropouts and disciplinary incidents in data reported to the state, as well as in data the state reported to federal authorities.

Auditors faulted the state for failing to verify the critical student information that flows from local districts and fuels policy decisions about Georgia's network of public schools.

But there is not funding for the level of auditing that would be necessary to verify data. In defense of districts, they have been asked to supply more and more data and in greater detail by both state and federal law without any corresponding raise in funding for staff to do so. Accurate data collection and verification is a major undertaking. 

Astropig
Astropig

@MaureenDowney @Astropig


I like having the info,I like having the various ratings.They're not written in stone,they're not the gospels.They provide a general picture of some things that need a little sunlight.

ErnestB
ErnestB

@MaureenDowney @Astropig


A good example is the introduction of the electronic gradebook.  Part of the rationale is that the time spent inputting grades in this would be offset by not putting them in the paper gradebook many of us grew up with.  


What administrators did not foresee is that expectations of timely entries grew exponentially.  State, central office, local school house and most of all parents wanted to see inputs almost immediately after some type of measurement was done.  If teachers did not have it in, they could get calls and emails from those in their reporting chain and parents, which takes time away from lesson preparation and instruction.  Could you imagine being in a school with many helicopter parents that expect emails to be returned in 4-6 hours?


Add this with the other requests being made of teachers, it is easy to see why their level of frustration has been increasing.  All this without an increase to the length of the school day and year.


I should point out the increased visibility of this data is good however at what cost to the teacher, especially with class sizes as they are today.

class80olddog
class80olddog

@MaureenDowney @Astropig So when the feds audited and found undercounts of dropouts and discipline issues, how many administrators got fired?  That is what I thought.  You can identify a problem all day long, but at some point you have to take corrective actions.  How many school systems still cheat on testing and just get away with it?

ErnestB
ErnestB

@class80olddog @MaureenDowney @Astropig


How do you know corrective action was not taken?  Personnel actions are typically handled privately.  Publicizing the results of personnel actions would result in more lawsuits and less money for students.

redweather
redweather

@Astropig  I don't think collecting data and using it is a bad thing, but that isn't exactly what this blog post is about. Rather it wonders if too much time is spent on data collection and not enough on helping students succeed.

speccie
speccie

The truly damning data are the many documented failures of public schools and their unwillingness to reform.

EdJohnson
EdJohnson

“In 2017, APS is investing $24M in a turnaround strategy for low-performing schools. APS is committed to making students career and college ready, and expects to continue significant investments in order to improve student achievement.”

Investing in making, huh.  As in investing in compelling; behaviorism, yet again.  How sad, and unnecessarily costly.  But not to worry.  APS’ teaching children Social and Emotional Learning will counteract making’s effects.  And if SEL doesn’t do it, then the APS Police Force will do the making.  So, see, all bases covered.  Students made career and college ready.

Kathy Brown
Kathy Brown

hahaha, it gives somebody a job doesn't it? The data helps to establish academic pecking order and gives systems bragging rights among their peers, which they go up to the state for "atta boy" recognition. Interestingly enough, there is rarely a conversation about the TWO education systems. One that has "certain federal, state, and local guidelines/policies, Title I schools, and non Title I schools. .

Tom Green
Tom Green

What? The state is supposed to be helping schools?

Wascatlady
Wascatlady

Maureen, when I looked at the database on the financial efficiency over the weekend, it had systems that did not seem to have a score.  Has this been corrected?


Pretty soon we will have efficiency ratings on bathroom usage--flushes and TP use per child.


What I would like to see is an accounting of administrators per child, perhaps by category.  I understand many posters' frustrations when there is a superintendent, 15 associate sups, each of them has 12 assistant sups, then district directors, associate district directors, and assistant district directors, each with subcategories for grade level.  Let's take a look, per system, at how many employees (outside of people like bus drivers, lunch ladies, janitorial staff) who have NO direct student instructional contact full time every day.  Now that is some data that would be revealing! CO staff, principals and their assistants, counselors, nurses, etc would all count as non-instructional (not saying they are not important). Perhaps counselors, nurses, parent involvement aides, social workers, instructional coaches, IT people could be a separate category.


Back when I first started teaching, the CO for a 3,000 student school system consisted of a superintendent, a secretary, a bookkeeper, and a 1/2 time visiting teacher/truancy person.  5 years later they added an assistant superintendent/HR person.  These folks worked hard!  Each school had a principal and a secretary.  The high school also had a counselor.  Other than bus drivers, lunch ladies, and janitors, that was IT!


Yes, there are economies of scale, but when your scale gets to the point that kids are a faceless number, your "economies" don't mean a &%$# thing!  Many systems in this state are like this.  It makes no sense in cramming in as many kids as possible to "save money."  SOMEWHERE along the line, it will cost you BIG TIME!

Tracy Marie
Tracy Marie

I don't even need to read this article... Yes! I can't think of a single thing the state has ever done that has helped a school.