Baseball and teaching: Why simplistic measures of performance strike out in both professions

University of Georgia professor Peter Smagorinsky compares baseball and teaching in this essay. He explains how judging winning performances in both professions defies conventional measures and simplistic approaches.

If you have time between innings of the World Series tonight, take a look.

billjamesBy Peter Smagorinsky

With the World Series upon us, let us turn to baseball. In “Moneyball,” author Michael Lewis provides a close look into the management of baseball teams, in particular how scouts and executives evaluate players. Traditionally, baseball men have relied on three offensive statistics—batting average, home runs, and runs batted in—as the three principal indicators of production and value.

In the 1970s, however, Bill James began questioning whether those three statistics indeed provided the best measures of performance. To illustrate his doubt, he gave an example that I will attempt to reconstruct from ancient memory, providing some more modern embellishments, such that details will vary. James’s example concerned two outfielders, each of whom hit .250, hit 25 home runs, and had 100 RBI. Based on these “Triple Crown” statistics, baseball traditionalists would view them as equally productive.

Let’s look a little deeper, said James. The first player, Goofus, struck out 100 times and walked 25 times; the other, Gallant, struck out 25 times and walked 100 times. Goofus hit into 50 double plays; Gallant hit into 5. Goofus is a left fielder, the outfield position for the player with the weakest arm; Gallant is a center fielder, one of the most important positions on a defense. Goofus played for the Rockies in Coors Field, baseball’s best hitter’s park, while Gallant played for Seattle, home to baseball’s best pitcher’s park.

And these are just offensive statistics; increasingly sophisticated defensive statistics further contribute to the evaluation of a player’s contributions. No single statistic predicts value; the aggregate, however, can profile quite accurately which players will provide a return on investment, barring injury or other less predictable variable; and which players are likely to perform below hopes and misplaced expectations.

The problem with traditional baseball statistics, according to Bill James, is they do not predict the kinds of production that produce wins. Over time, his field of sabermetrics has provided far more sophisticated means of measuring a wide variety of factors that add up to suggest a player’s value. They have found some statistics, such as batting average, are of far less value than more nuanced measures such as on-base average. James and colleagues made the case that if you paid Goofus the same as you paid Gallant when they were free agents, based on their deceptive Triple Crown stats, you’d be overpaying for Goofus and underpaying for Gallant.

The proof of this approach’s success, to Michael Lewis, was in the poorly funded Oakland Athletics’ ability to compete for playoff spots against the league’s financial heavyweights year after year. By measuring the right things instead of the most obvious, a growing number of baseball executives such as Oakland’s Billy Beane were able to invest with savvy in undervalued talent and in players with selective, productive abilities in order to produce a winning team.

I am thinking of more than just baseball here. I am thinking of simplistic means of measurement in education. I won’t belabor the point, which so many of us make so often in this space, but I’m thinking in particular of the use of test scores as the sole means of measuring student performance and teacher effectiveness.

A batting champion who bats .350 but rarely draws walks is of less value than a player who hits .300 and draws 100 walks, because as the old adage says, a walk is as good as a hit — even though the league’s on-base percentage leader rarely gets the accolades that await a batting champion.

In classrooms, many factors go into being a good student, almost all of which are not counted in today’s accountability movement. In baseball, the quality of the Oakland A’s over time suggests measuring the right things can produce a winner at a low investment cost. Other teams undervalue what actually produces a win, falling back on statistics identified over a century before computers and electronic record-keeping made better means of measurement available.

Education is in a similar state as baseball right now, although with some differences. Standardized testing — one of many means of evaluating learning and teaching, and by many accounts among the easiest to reduce and most unreliable to employ — has highly sophisticated data bases and statistical means available to its supporters, including the widely discredited “value-added measurement” of teacher evaluation. Yet computers, no matter how sophisticated, are used to measure what’s obvious instead of what matters.

Some might argue the tests are reliable because they tend to arrive at the same conclusions about the same students over and over. Politicians and policymakers love such simplistic thinking. The problem is that they are measuring what’s most easily measurable, not what education is conceived to accomplish.

I wonder what would happen if we do as sabermetricians have done with baseball to identify what really matters in measuring performances. In baseball, winning is the primary goal, and so it is different from education. Education needn’t be concerned with winners and losers, even though the current policymakers appear to think in terms of races to the top and other competitive metaphors that guarantee that when tests are normed, half of the population (and their teachers) will be labeled as underperforming.

Op-ed and blog essays are too brief to solve all educational conundrums; if you’re interesting in a detailed proposal for authentic evaluation published in a peer-reviewed journal, see this. Here, I would simply say educational assessment would benefit from doing what baseball sabermetricians have done and find a wide range of indicators that contribute to profile a learner’s academic growth.

Baseball statisticians, for instance, look at “park effects,” the degree to which playing within a ballpark’s particular contours affects the statistics a player produces when on that team. Spacious parks depress power statistics, for instance.

In education, as so many have demonstrated, poverty depresses test scores and other measures of academic achievement, and for many reasons: hungry kids are prone to illness and have a difficult time concentrating on abstract schoolwork, they miss school and move frequently, they have few familial models of academic investment and success, they often grow up within a climate of distrust of public institutions, and so on. That’s not an excuse; that’s a fact.

Rates of poverty and affluence could serve among the many factors that thoughtful educators could take into account given the power of computers to parse out phenomena statistically. Many authors of essays in this space, including me, have written extensively here on more authentic conceptions of teaching and learning that could help round out the misleading profile available through test scores alone.

But it’s much harder to think of how to value complex performance than it is to stick to tests developed ages ago for other purposes to provide a single number that purportedly encapsulates the whole of human growth and achievement. That’s what kids are learning in school these days: Teaching and learning are simpleminded. I can’t imagine that disposition to be of much value when they enter the world of work and find that it just ain’t so.

 

Reader Comments 1

34 comments
priyanshu beautique
priyanshu beautique

Shri Krishna Janmashtami is the Hindu festival celebrated every year on Krishna paksh ashtami of Bhadrapada month according to the Hindu calendar. The day is also known as gokulashtami, saatam aatham, rohiniashtami etc. Janmashtami is the celebration of Lord Krishna's birth that was eighth avatar of Lord Vishnu.

for more

Plz visit:- <a href="http://www.dooiitt.com/category/indianfestivals/janmashtami/">Janmashtami 2017</a>


siya patel
siya patel

Happy Independence Day, India - the "land of the free, and the home of the brave"! How well do you remember August 15th? Ask a youngster about the significance of Independence Day, and on that day prime minister of india give speech at red fort.

Please visit<a href= "http://www.dooiitt.com/category/indian-festivals/independence-day/">Independence Day Speech</a>

siya patel
siya patel

As India celebrates its 70th Independence Day on 15 August 2017, its citizens rejoice celebrating a different kind of freedom - the freedom to travel, explore, and unwind. The Indian citizens are all perked up for a much awaited long weekend during Independence Day.

Please visit <a href= "http://www.dooiitt.com/category/indian-festivals/independence-day/">Independence Day</a>


siya patel
siya patel


Independence Day is celebrated differently and on difference dates around the world. There are fun traditions and ways to take this holiday. Find ideas about Independence Day Images, Quotes, Speech, Wishes, Status and Celebration,etc. For more...

Please visit <a href= "http://www.dooiitt.com/category/indian-festivals/independence-day/">Independence Day Images</a>

Dooiitt Tube
Dooiitt Tube

Relationships are the essence of any festivity and it holds true for any Indian festival. Each occasion brings the family together which 

calls for a celebration. Raksha Bandhan is a celebration of one such relation - that of a brother and a sister. 

For more…

Plz visit:- <a href="http://www.dooiitt.com/thread-love-raksha-bandhan/">raksha bandhan</a>

Dooiitt Tube
Dooiitt Tube

Relationships are the essence of any festivity and it holds true for any Indian festival. Each occasion brings the family together which 

calls for a celebration. Raksha Bandhan is a celebration of one such relation - that of a brother and a sister. 

For more…

Plz visit:- <a href="http://www.dooiitt.com/thread-love-raksha-bandhan/">raksha bandhan</a>


Dooiitt Tube
Dooiitt Tube

You can make the different different type of rakhi like lion rakhi and lion rakhi replace with chota bheem,spider man,pokemon sticker,

you can also make flower,peacocks,geometrical designs,photo rakhi idea from picsquare.

For more…

Plz visit:- <a href="http://www.dooiitt.com/rakhi-making/">rakhi making</a>

Dooiitt Tube
Dooiitt Tube

Best Outfits For Young ladies,Womens and Girls

We Have Some For You In Your Budget For more…

Please visit:- <a href= "http://www.dooiitt.com/"> Designer Salwar Suit</a>

Lee_CPA2
Lee_CPA2

Wait for it.....wait for it......

"In education,  poverty depresses test scores and other measures of academic achievement."

Bingo!  There it is.  The politically correct educrat's excuse for most things wrong with education.

Hey Smaggy, here's a stat that juuusssstttt might  be relevant when discussing adademic achievement - Intelligence Quotient.

Or, we could just give everybody on welfare $1 million.  That would solve the economic academic achievement disparity, I'm sure.

Joney
Joney

I found the analogy in the article to be valid and persuasive.  My only objection--not to the article, but to looking at multiple measures--is that you can't measure human quality.  Could anyone  propose numbering their familly members or friends to show whether or not they qualify to be admitted to Heaven afterleaving this earth?

redweather
redweather

@Joney "Could anyone propose numbering their familly members or friends to show whether or not they qualify to be admitted to Heaven afterleaving this earth?"

I could do that. :-)

BearCasey
BearCasey

I don't know how this article became hi-jacked onto the well worn path of class warfare.  That's not what I got from it at all.  It's a plea to develop accountability measures in addition to traditional measures. For example, an important factor I almost never see mentioned here is how much a teacher is able to inspire love for learning, creating lifelong learners.  Great students continue to learn long after they've finished formal schooling.  One would think we would want to identify teachers who promote THAT.

Astropig
Astropig

I guess this is what liberal educational theory is reduced to these days- Class warfare and outright deceptive reasoning to keep the game going a little longer.No wonder reform movements are becoming a tidal wave.


If this is all you got,Peter, you ain't got much.

redweather
redweather

@Astropig Care to explain how his reasoning is deceptive, or are you going to wait for your friends Frick and Frack to do that?

redweather
redweather

@Astropig @redweather Just interested in knowing how his reasoning is deceptive.  That's a pretty strong charge. Faulty I could see, but deceptive?

Don't Tread
Don't Tread

"Rates of poverty and affluence could serve among the many factors that thoughtful educators could take into account..."


Here we have class warfare and social promotion neatly wrapped into one paragraph.  So I suppose then if an educator decides not to socially promote a kid based on poverty, they're not "thoughtful" enough.


I suppose a "thoughtful" educator could also add other things to the magic computer model like: no father in the house, family members with criminal histories, drug use by adults the child comes in contact with, etc.  I wonder how that math would work.


Oh wait...no I don't.  I know exactly how that would work...penalize kids who are brought up right and are doing the right thing, because they would have too many things "in their favor".


This is exactly why liberals are running the public education system into the ground.  "Thoughtful" parents will send their kids to private schools or homeschool them.

Astropig
Astropig

@DontTread 

I don't know about you, but I'm sick of class warfare. Fed up.

living-in-outdated-ed
living-in-outdated-ed

@DontTread my one issue is the last sentence. Impoverished families don't have this option, but they could send them to public charter schools if an option existed.  So you do need to be very careful when you talk about public education.   Every child, regardless of socioeconomic circumstances, should have right to a quality education.  But again, for the millionth time, our state constitution uses the words "adequate education."


Why do we keep forgetting this?????  Don't we all think the constitution needs to be amended for such an obvious syntax error?

Don't Tread
Don't Tread

@living-in-outdated-ed @DontTread If there were vouchers, they'd have more options.  But the liberals currently running the public education monopoly are adamantly opposed to vouchers...surprised?

living-in-outdated-ed
living-in-outdated-ed

Absolutely horrendous analogy, Peter.  Sounds like something that Anthony Cody or Paul Thomas would author. Try and force an analogy that is timely and identifiable, but in the end, a crude analogy at best.


The Oakland A's look at lots of data, but they will always struggle to retain free agents because of the size of their market.  They churn out talent all the time.   They keep moving pieces in and out and they really don't have any team loyalty.  Most teams don't have players that spend their entire careers with one team.  Those are few and far in between a a relic of a generation long since past.   Ahh - just like education - living in an outdated system.


There is such a thing as analysis paralysis, and the problem is - all of our accountability formulas, even the QBE, have been identified by third party think tanks as too complex.   You have to have some accountability data, because data can be used as a predictor.  But the problem is, we are overemphasizing quantitative measures and the accountability formulas are not balanced enough.  We've had decades to figure this out, yet we still can't get it right.  Even in baseball, it's all about probability, Peter.   There will still be plenty of anomalies.   Just look at these Royals!   A complete anomaly.  They don't even play an "American League" brand of baseball.


So lets throw your baseball and sabremetrics analogy out the window, especially the poverty analogy.  And we don't have to debate poverty again.  It's a crutch, and we know that there are certain charter schools and other environments who are able to teach impoverished kids.  Again,  I will mention the Atlanta Music Project ("El Sistema") and Paul Tough's book to emphasize such opportunites.


Bottom line - bad analogy, Peter.  And sorry if we agree to disagree.


Go Royals!

Astropig
Astropig

@living-in-outdated-ed 

" Go Royals!"

Amen.The Royals are a pretty good opposite example of whatever weird point this article is trying to make.Not that long ago, they were by far the worst team in baseball. (They lost 106 games in '05 and 100 loss seasons were common).Instead of trying to make the other teams in MLB worse so that everything was "fair", the Royals realized that what they were doing wasn't working and made themselves better.The results were not immediate,but they have paid dividends.


I hope they score 10 runs again tonight and pitch a shutout(again).

straker
straker

Professional baseball has been around in America for close to a century, at least.


I seriously doubt a UGA professor knows more about how to evaluate performance than seasoned managers with years of past data to rely on.

BearCasey
BearCasey

@straker  The whole point was that a relative nobody (at the time), Bill James, DID know more than the "seasoned managers" did and the Oakland A's great performance was the evidence.

Looking4truth
Looking4truth

The main issue I have with this analogy is that Mark Richt, Freddy Gonzalez, etal do not have to take every player that comes to the field.  They are very selective when it comes to who gets to play for them.

Give any teacher the same right (and the same pay) and watch the chosen few succeed.

dcdcdc
dcdcdc

Its said of Bear Bryant that "he could take his'n and beat your'n, and he could take your'n and beat his'n".

That's the mark of a truly great coach, manager, teacher/professor, or leader.  Now...reality says, most of us just aren't great.  But for those who are, the idea of paying them "the same average salary" as the rest, is truly insane.

Someday, we'll have special schools with special teachers, who are paid "obscene" amounts of money (but who are fired if they don't perform, just like coaches).  Those schools will blow away everyone else, regardless of the quality of the incoming students.

class80olddog
class80olddog

@dcdcdc  BS!  You take Bear Bryant and tell him to win with players who won't show up for practice and he can't kick them off the team?  He would have told you you're crazy - he has to have the AUTHORITY to make kids do what he wants or kick them off or all bets are off.

class80olddog
class80olddog

The baseball analogy is a very poor one, but consider this - the coach is required that every player hit a .600 average.  If he does not achieve this he will be on a performance improvement plan and subject to firing  He also has to do this with players who don't show up for practice and he cannot discipline such players.  Sound good?


EdUktr - accountability is good, but only for those things that a teacher has control over.  Where is the accountability for the principal for attendance?  If a certain level is reached, is he/she place on a PIP? Where is the administrators' accountability for discipline and social promotion

SouthGeorgiareader
SouthGeorgiareader

The Doc is right.  Nothing involving the human brain and how it works and responds to outside forces is simple, even though we persist in trying to make it seem that way.  Standardized testing is itself just a simple way to evaluate, missing the finer points of teaching and learning.

EdUktr
EdUktr

Oh, God. Yet another sermon on the "unfairness" of testing and accountability. To a wider audience baffled by your inability to accept the absolute necessity of both.

Starik
Starik

I agree that simplistic performance measures don't work.  The baseball/teaching analogy won't work either.  If baseball teams were assigned different balls to play with, though, the analogy would be better - if one team played with a nerf ball, one with a soft ball, one with a baseball, one with a tennis ball.  Teachers in most DeKalb schools and teachers in North Fulton have balls with wildly different characteristics. Graduation would be like the on base percentage or batting average with a baseball in DeKalb, a soft ball in Decatur or North Fulton. Or maybe a tennis ball - the batting average is high but home runs would be a rarity.

Trackbacks

  1. […] in both professions defies conventional measures and simplistic approaches in an October 28 ​Atlanta Journal Constitution op-ed […]