Does Georgia now have a teacher evaluation system only a sadist could love?

This is a lengthy account of Georgia’s new teacher evaluation system by a teacher in a Georgia school system.  Rated as highly effective in the past, the teacher helped develop the End of Course Test in his discipline.

In this piece, the teacher reports on a recent training session in which the presenter said teacher evaluations last year were too high and administrators were being taught how to downgrade their ratings. The teacher also shares his experience with the new Milestones test.

He provides a fascinating and detailed look at an evaluation system that seems inconsistent, arbitrary and ultimately self-defeating. This approach does not seem designed to improve Georgia teachers, but to drive them out the door.

I am sharing it at a longer-than-usual blog length as it helps explain why teachers are concerned about linking their evaluations to student test scores, as sought by Gov. Nathan Deal.

Here is the teacher’s account:

Recently, my colleagues and I were introduced to a “Deeper Look at Instructional Practice and TKES.”  The two-hour presentation started with a short history of TKES and how it originated with House Bill 244 during the 2013 legislative session.

The law creates a single, statewide evaluation system for teachers of record (the teacher whose name appears on the transcript as the teacher of a particular class).  The law requires student achievement be tied to the teacher’s professional evaluation. Student achievement must contribute at least 50 percent to the teacher’s Teacher Effectiveness Measure or TEM score. The law requires that student achievement be weighted more heavily than observations.  The purpose is to optimize student learning and growth, improve the quality of classroom instruction, and support the continuous growth of teachers.

This diagram was provided to explain how a TEM score is calculated.  Teachers are scored on one of four Levels I, II, III, IV, with IV being exemplary:

evalchart

A quick glance at the chart shows there only seven boxes in which a teacher is considered “proficient” or “exemplary.”  There are nine boxes in which a teacher might be considered “needs development” or “ineffective.”

A question was immediately raised by a teacher: “How are we supposed to succeed when there are more opportunities to do poorly than to do well?”

The response of the presenter: “I advise you to contact your legislator, as I am only the messenger.”

The conversation quickly moved on to the real reason for our meeting that day.  There was a general lack of consistency for the 2014-2015 school year between ratings for teachers and ratings for student achievement.  Teachers were consistently rated at Level 3 or proficient.  But, student performance on standardized tests was on average at Level 2 or needs development.

The state Department of Education wants teacher ratings and student ratings to align. Therefore, the solution was to teach administrators how to rate teachers lower. Rather than address the reasons why students do poorly on tests, the quick fix is to downgrade the teachers who clearly failed to do their jobs.

Administrators were trained over a three-day period earlier in the school year, so it was only fair that teachers be informed of what to expect. “It wasn’t economically feasible to train every teacher in the state of Georgia on the TKES platform, so the state decided to teach the raters (administrators) first and worry about the teachers later,” said the presenter.

To quote the presenter, “We are going to build the plane while we fly it.”

And finally, “The state wants to make sure we get this right, so we pushed back the full implementation of TKES until 2016-2017.  Your results won’t begin to count against you until 2017.”

The point is raters (administrators) need to give more 2s in their ratings of teachers and fewer 3s  Teachers should shoot for Level 3 in their practice, but the reality is that you can’t be there all the time. Teachers shouldn’t take it personally when they are given 2s on their observations, but rather see it as an opportunity to grow in professional practice.

One teacher asked, “Is it possible to get a Level 4 (exemplary rating)?”

The presenter responded: “The language of the standards states a Level 3 teacher consistently performs a certain action, and that a Level 4 teacher continually performs that action.  So consider a medical analogy: If you take a vitamin daily, but you forget to take it on vacation or the occasional Sunday morning, then it’s safe to say you take that vitamin consistently, but the only real way to take it continually is to be hooked up to an IV bag.”

Several other questions were asked but the answers were all similar: No system is perfect; they don’t really count against you yet; contact your legislator.

The remainder of the meeting was divided into two parts.  There was an in-depth presentation on TKES standard 4: Differentiated Instruction, and standard 3: Instructional Strategies.  Finally, the presentation addressed how raters would collect evidence of teacher effectiveness and how administrators were trained to share the results with teachers through “mediated thinking” strategies.

The keys to addressing the TKES standards 3 and 4, according the presenter, were in lesson planning.

A lesson plan form was shared with us that would be the delight of any college professor.  In short, we needed to write into daily plans how the lesson was differentiated for the different ability levels in our classroom and provide administrators with a “look for” guide as to our efforts to address all learners. If the administrator is not provided with a “look for” list, then the highest a teacher could be scored is Level 2.  The thinking was that if you can’t show it up front, you don’t do it.  All this in addition to lesson plans to reference GPS standards, lesson content, etc.

Also, all lessons will now be required to “engage students cognitively.”  The presenter explained the difference between “compliance” and “cognitive engagement” as this: “Students sitting in desks in rows, with books on their desks, doing seat work, following the lead of the teacher are simply engaged in a compliance activity and are learning nothing. This can be scored no higher than Level 2. Students who are active, talking, moving around the classroom, working in groups, or engaged in hands-on activities are the ones who are cognitively engaged.”

That seemed plausible to me, but I kept thinking back to the earlier analogy regarding a level 4 rating.  A teacher asked, “What about cognitively engaging students who stayed up all night, came to school on drugs, or just don’t care about school?”  The answer was, “If you teach at the highest possible level, then the test scores and everything else will take care of itself, you won’t reach every student.”

The collection of evidence didn’t seem out of the ordinary to me.  Just like college professors who observed me as a student teacher back in the 1900s the administrators are asked to script what they see throughout a lesson and to remove bias from their observations. Removing bias seems difficult, if not impossible, but the state DOE has ordered it, so it shall be done.

The final few moments of the meeting dealt with how an administrator is supposed to kindly share with a teacher the downgraded marks they are to receive.  The process is called “mediated thinking.” The process involves several steps: The administrator will remain neutral during the evaluation meeting.  Collected evidence is to be used as the “third point of conversation.” Use this as an opportunity to “shine a light” on the teacher’s thinking about instruction and evaluation and teach them the proper way to view them.  Push teachers to self-reflect on their practice and move themselves from (Needs Development) to (Proficient) and from (Good) to (Great).

As a teacher, I would like to offer some thoughts/notes of my own to what I saw that day.  My first exposure to TKES came in July of 2014 when all teachers were given an orientation by the system director of curriculum.  We were informed student growth was paramount to any individual teacher’s evaluation by an administrator. Teachers were going to be graded on a four point system.

A Level 1 will get you fired. A Level 2 will get you put on probation and another Level 2 after that will get you fired. 

Most teachers live in Level 3 land and sometimes they visit Level 4.  Do not expect 4s and shoot for 3s.  Hopefully, this will go away before anyone gets fired. I survived year one with all 3s and 4s.

Fast forward to this year and the message seems to be there were too many 3s and not enough 2s and 4s.  Schools can be rewarded for giving teachers 4s if student achievement matches that rating, but we won’t know that until after the fact.  It sounds like the new normal is to expect 2s.  But I think back to summer of 2014 the message was too many 2s will get you fired.

Student achievement tied to test scores is a real problem. I know a bit about the state’s End of Course Tests. I worked on the creation of my subject area’s EOCT.  I worked as a representative of the Georgia Department of Education with Pearson Educational Measurement and attended more than 20 meetings in Atlanta that usually lasted three days at a time. We developed the testing specifications, divided the testing domains, looked over 8,000 multiple choice questions word by word.

We looked at data from questions that were field tested and kept and rejected questions based on that data. Finally, we normed test data and gave Pearson a benchmark by which to score every student in the state. This was an eight-year process beginning to end.  I was there every step of the way, usually serving on teacher committees of 12 members or fewer. I used what I learned to my advantage, and I consistently had the highest EOCT scores in my county from 2004 until 2014.

I’m not saying I used information illegally or unethically, but when a teacher knows the test inside and out it’s pretty easy to teach a student how to play the game.  Sometime in 2014, Pearson Educational Measurement lost its Georgia contract and was replaced by CTB/McGraw-Hill.

I was immediately chosen by them to serve on the same types of committees I had served on before.  I learned the new Milestones test for the first few years would be identical to the tests that Pearson had given because McGraw Hill didn’t have field tested items yet so they couldn’t create their own version of the test.

The EOCT created by Pearson was property of the state of Georgia so they would use those questions for the 2014-2015 EOC and just grade it to a new set of criteria.  I was thinking no problem, same test, same questions, my students should do well.

When last year’s scores finally came out in November I was shocked my pass rate had dropped from the usual 90 percent or higher to 29 percent.  I had changed schools in between, same county, but I didn’t feel like my new students were that much worse off than any other group I had ever taught, and surely my teaching ability didn’t decline that much in one year.

What I noticed was that in order to pass the Milestone at a proficient level a student had to score 80 or higher. Students who scored in the 70s were considered “needs development” or “Level II.”

In the past students were rated as Exceeds Expectations, Meets Expectations, or Does Not Meet Expectations. Now students are rated on a 4-point scale just like teachers, and, just like teachers, a Level 2 is not a good thing.  The real problem with the test scores is that a student can pass my course and receive the Carnegie unit for a score of 70 or higher.  But, my rating as a teacher will suffer if the same student doesn’t score above 80.

I had one student in particular that came in early every day for two weeks leading up to the EOC last year.  She was a solid B student but she was worried about the EOC.  I worked with her and she pulled out a 79 on the EOC.

The EOC counting as 20 percent of her final average was waived since scores did not come back in time, but even without the waiver she would have kept her B average in my class with a 79 on the EOC.  The results from that situation are the student is happy she kept her B, she performed as well as she was ever going to on the EOC, and the teacher receives a rating of Level 2 because she didn’t score 80 or higher.  It’s scary to think that 50 percent or more of my evaluation depends on that number considering it was the same test students have been taking for a decade, it’s just graded much tougher now.

I’m no expert in mathematics, but statistically I know most people are average, and I know the bell curve says that most students will score around 75 on any valid exam.  It seems unfair to move students ahead in the system for being average, but to punish the teacher when an average student doesn’t score above average on a standardized test.

Reader Comments 1

145 comments
32LM
32LM

The problem is not only the legislation.  The biggest problem is the way the school system has implemented it.  Once again a bureaucratic paperwork drill to cover the administrators rear ends and blame the teachers when things go awry.  

AverageParent
AverageParent

It doesn’t make since that a teacher is evaluated by one person based on a 20 minute observation…Principals and admin don’t know what really goes on in the classroom “regularly” and if your evaluation is done when the kids are having a bad day, you get scored and evaluated based on that one 20 minute observation.  Other teachers should be brought into the mix and have more than one observer in the class room to balance the evaluation process out (you know- since teacher know what teaching is really like).  The students are disrupted by having a visitor in the class, the teacher is nervous that they are being observed…just totally wrong.

If teacher are going to be measured by students learning, then a benchmark measurement from day one in that teachers class needs to be established, otherwise the teacher is responsible for the learning all other teachers gave that student before they got them.

drdon
drdon

Fall 2015 PAGE (Professional Association of Georgia Educators) survey of 6200 teachers -

   68% would not recommend a career in education.

The state "needs more STEM graduates to attract companies" yet a new h.s. math teacher has 170 students. 

advocateforchange
advocateforchange

The biggest problem with TKES is the lack of consistency in evaluations.  


If the administrator likes you then your evaluation will be good.  


TKES is abused frequently.  Many teachers have leaved the profession due to their TKES evaluations.  


JBBrown1968
JBBrown1968

126.000 people standing at the capital on a school day is the only thing that will change Georgia education. Being job scared and using the victim card is killing the teaching profession and education! 

Dr. John Trotter
Dr. John Trotter

What I wrote on this topic, I posted on www.georgiateachersspeakout.com.

Dr. John Trotter
Dr. John Trotter

Thanks, Catlady.  I see that the Blog Monster remembers me because he apparently ate my post.  LOL!  I do use a language that is a bit spicy but I tried to encrypt it.  LOL!  I have been busy writing books in recent years and doing my own blogs...but it is good to check in occasionally.  Maureen and the AJC do a great service.

Dr. John Trotter
Dr. John Trotter

Maureen, I tried to post a fairly lengthy comment about TKES and holding teachers accountable for a significant number of students who simply refuse to learn...and also about holding them accountable to these students' evaluation-retaliation of the teacher, but I don't know if the comments ever posted or were simply grabbed by the blog monster.  Oh well. 

Wascatlady
Wascatlady

I suspect one day, long after I am gone, teacher evaluation will rest on what the TEACHER has done.  Has s/he given the student an excellent opportunity to learn the material?  This won't involve moving actual mountains, or adopting members of the class who don't have parenting persons into the teacher's home for the year.  It will involve planning and delivering and evaluating instruction. It will involve attempting to bring in others to address problems present in the child's life--counselors, parents, DFACS, whatever.  It will NOT involve judging teachers for what OTHERS do or do not do. If the state, or the school system, refuses/neglects to provide programs, policies, or personnel to aid in student "recovery,"  it will not be used to downgrade the teacher.  If the student does not do their part (attending, paying attention, working to learn) the teacher will NOT be downgraded, but the student will receive the results of their lack of effort.


Evaluate a teacher on what s/he does--planning, delivering, and evaluating instruction on the curriculum, and advocating for the student who needs additional help--and I doubt many teachers would object.

cfhogan
cfhogan

I have been retired for 10 years, however I taught for 18 years starting at the ripe old age of 39.  Then they had the TPAI (Teachers Professional Assessment Instrument) for new teachers.  The colleges were forced to teach it and you had it hanging over your head and it was pages and pages long.  They need to get away from all this time consuming mess and let the teachers teach and administrators administrate.  The principals know who can teach and who can't.  You don't need all this evaluation.  Think of all the money that could be spent in the classroom and the smaller class size if we weren't so busy testing and evaluating.

CSpinks
CSpinks

That seldom does any teacher who posts on Maureen's blog provide a legal or readily-identifiable name speaks volumes about the climate of intimidation and fear which grips too many of my currently practicing colleagues.

redweather
redweather

@CSpinks You are correct, although I don't believe the fear is well founded.  Rather, I think it's a cop-out.

Wascatlady
Wascatlady

@AvgGeorgian @CSpinks An Elected Official Reform Commission without any elected officials.  HEY, that actually sounds like it might help the taxpayers!

AvgGeorgian
AvgGeorgian

@CSpinks


Deal created the Education Reform Commission, appointed all 34 members, and not one of them is a teacher and currently teaching.

Maybe some other ‘commissions” will be created this year:

A Law Reform Commission with no lawyers

A Dental Reform Commission with no dentists

A Medical Reform Commission with no doctors

A Plumbing Reform Commission with no plumbers

An Engineering Reform Commission with no engineers

An Automotive Repair Commission with no mechanics

Heck, the same commission members could serve on all the commissions.

CSpinks
CSpinks

@redweather @CSpinks (R)edweather, you obviously don't teach in an east-central Georgia public school system. Corroboration for the credibility of my comments can be obtained from Dr. Wayne Frazier, a retired Richmond County School System administrator, at frazier27@comcast.net and from John P. Batson, a civil rights attorney practicing in Augusta, whose e-mail address is jpbatson@aol.com.

Starik
Starik

Educate me: What role do Principals play in hiring teachers?  Does it vary by district?

Scott Kent
Scott Kent

@Starik It varies, but in most Greater Atlanta districts, the county approves them to be hired within the county, but then the principal makes the final decision about whom to hire.

Starik
Starik

@Wascatlady @Scott Kent @Starik Well, if I were Emperor of the Schools I'd consider this:


1. You have to test the kids. That gives you objective numbers that will withstand litigation. Test at the end of the year, and upon arrival for those whose who arrive during the school year.  Devise a common sense measurement of what students learn from their teacher. 


2. Don't rate the teachers with the byzantine scheme described above. Rate the Principal based on the test scores, adjusted for poverty, SPED and so on.


3. Let the Principal decide who gets raises based on merit, and let them have full control over hiring and firing. 


      Put the pressure on the Principal. 

JBBrown1968
JBBrown1968

Politicians are running education! Let them report for duty! 

newsjunkie523
newsjunkie523

@JBBrown1968 Exactly. They are legislating the teachers when actually the students and families should be legislated. There are many outside student factors teachers cannot overcome.

JBBrown1968
JBBrown1968

Stop whining on this blog and organize! All of you can afford a few sick days to solve this continuous debate! Blue flu for the betterment of Georgia Education! GAE, PAGE, and MACE are jokes!


heyteacher
heyteacher

I've taught under multiple evaluation instruments and put TKES up there with the TPAI (which was notorious for failing teachers on something as inane as not having the correct date on the board during an evaluation). I predict that TKES will go away once the testing component is part of the evaluation and so many teachers are "failing" -- there's so much ridiculousness to using test scores as part of the criteria that I don't even know where to begin. A wise admin once told me to just keep teaching the kids and not worry so much about the adults in this equation .... just close your door she said. And so I shall. 

redweather
redweather

Absurdity #1:  "we needed to write into daily plans how the lesson was differentiated for the different ability levels in our classroom and provide administrators with a “look for” guide as to our efforts to address all learners."


Absurdity #2:  "Students sitting in desks in rows, with books on their desks, doing seat work, following the lead of the teacher are simply engaged in a compliance activity and are learning nothing."

redweather
redweather

@Wascatlady @redweather I am assuming that compliance activity is viewed as tantamount to so-called busy work, but that certainly isn't always the case.  I've found it necessary at the college level to assign my students in-class work that forces them to do things they won't do on their own. One of the benefits of this, for me at least, is that I can claim I've "flipped" my classroom and facilitated "student-centered" learning.

Sandy4271
Sandy4271

Training? What training? Let me share with you what one administrator rated me just a couple years ago. I received a 2 on efficient use of class time management because I clicked on a folder to access a PowerPoint instead of having it running in the background. I received another 2 for classroom management because two kids in the corner closest to the administration had a brief off-topic discussion during group activities in which both my co-teacher and I were actively monitoring and helping other groups. My 3rd 2 (which I argued and was ready to contact my PAGE lawyer until it was changed) was because we were told to do remediation lessons for any concept that did not have a 50% mastery level. I did that but received a 2 because he said I should have done more than what he asked and been proactive and put the marker at 25%. When I told him that would make it a 4 and that he couldn't give me a 2 for doing what he instructed me to, he actually had the gall to reply that I should have followed the spirit, not the letter, of his instruction. This is what TKES and LEKS means to me and why I will probably be changing careers after this school year.

GA_and_Education_futile
GA_and_Education_futile

Teacher shortage coming to a Georgia school system near you. Thanks Maureen for this information.

I've had many occupations before getting into education.  A career in education is much harder than anything that I've ever worked, besides being a parent. It's difficult trying to motivate students to learn when their home life (their real day-to-day) says..."Whatever!"

newsjunkie523
newsjunkie523

@GA_and_Education_futile 44% of new teachers are quitting in the first 5 years. Student teacher enrollment is down 20%. I wouldn't encourage anyone to enter the field at this point. What they expect from primary and other students is unrealistic and not within their developmental ability.

HILUX
HILUX

I doubt there are many actual teachers who expect their profession to be the only one on earth in which workers aren't evaluated yearly.

And yet, the blog's malcontents would have us believe so.

ALW930
ALW930

We expect to be evaluated. But evaluate us on what we can control: our planning and teaching, our education, experience, and professional development. All students don't always engage, and some students and their parents refuse to participate. Google Vollmer's Blueberry Story.

Teachers take what we are given, and we do the best we can. Evaluate us fairly. And for parents who REFUSE to make their children behave, give teachers and administrators legal recourse to deal with PARENTS who are FAILING their children. The burden for teaching is on teachers. The burden for learning should be on students, and by extension on the parents. Until our society returns to this understanding and stops placing both burdens on teachers, education is not going to improve. And here's another thought: who is responsible for a child from birth until he or she is able to stand on his or her own?? The parents. Most teachers work with a student for a year. When a child is arrested, the parent will be the one called to bail him out, not the teacher or the school system. And when that child can't hold down a job, on whose couch will he be sleeping?? Yes, the parent's.

I teach my heart out, as do my colleagues statewide. The equation needs to be balanced, and right now in Georgia, and the US in general, it's not.

newsjunkie523
newsjunkie523

@ALW930 Well said! Yes, at failing schools they never adjust or account for the SES of the community or whether the parents are parenting. And this of course, will be where the shortages will be the greatest.

I3>Teaching
I3>Teaching

@HILUX I don't know any teachers who don't want to be evaluated. We welcome evaluations. We used to welcome administrators into our rooms when they could be there. What we object to is the monstrosity of this extremely flawed evaluation system.

AvgGeorgian
AvgGeorgian

@I3>Teaching @HILUX


Tell you what HILUX, you take the 30 closest(geographically) adults in a "failing" school zone as your workforce. 


Only 3 rules:


1. You can't force them to work

2. You can't pay them

3. You can't fire them


Get back to us on your evaluation.

BG927
BG927

@HILUX

@HILUX What on earth are you babbling on about? Who has said anything about teachers not being evaluated yearly? We are, and have been, at least as long as the 15 years I have been teaching (I still have the 15 evaluations to prove it).  As a matter of fact, up until last year, my first year under TKES, the older evaluations resemble very strongly the evaluations my husband received from his job at a major metro energy company, my daughter's from a major metro military subcontractor, and my son's from a major metro medical company.  The funny thing is that all of their evaluations are on THEIR performance, and the elements of which they are in control. Quit being disingenuous.


To put this evaluation system to a corporate comparison, imagine you are a sales manager in charge of 30 salespeople.  You can't fire them, no matter what they do (however, you can send them home without pay for a couple days if your boss okays it).  You do everything you are supposed to and then some: you follow your bosses' rules and recommendations for structuring their workday, motivating them, and assessing their performance.  You give them meaningful, timely feedback on how to be better salespeople.  If someone is really struggling, you might bring them in for extra help in cold calling or closing. Your weekly business plans are spot-on, and your bosses regularly tell you so.  As a matter of fact, your bosses are so impressed with your performance, they ask you to help other managers who might be struggling.  Your evaluations are excellent.


However, your salespeople aren't doing so well.  Maybe the product isn't in demand.  Maybe the product, through no fault of the salespeople, is shoddy, and getting bad reviews.  Maybe the company has made some bad business decisions, and your team is trying to sell your product in an already saturated market. Maybe HR hired some salespeople that aren't really cut out for the job - it doesn't matter; you can't fire them.  Your team's numbers are bad.


According to what this teacher wrote, you would get a poor evaluation despite your good performance.  Who in their right mind, in what profession, would sign up for that nonsense? And folks wonder why ed schools' number are down.

Whitney Allison
Whitney Allison

"But, my rating as a teacher will suffer if the same student doesn’t score above 80."  


I have been under the impression that our teacher rating (TEM) is based on whether that student grew, not his or her 80.  Fifty percent of the teacher's evaluation comes from student growth, not the lass percentage of 3s and 4s.

Wascatlady
Wascatlady

One other comment:  I got a Bachelor's, 2 master's, and a PhD largely by "not learning anything"with that dog-gone "compliant behavior!'

ALW930
ALW930

Compliant behavior makes it possible for children to learn. I have one particular grade level which is filled with students who have chronic behavior issues. During class, I get one student settled, and then another behavior pops up. I spend most of my time with that group putting out fires rather than teaching. And by and large, the parents are unresponsive. Teachers and administrators have done everything we can, so upon whom should consequences fall? I propose legislation that imposes severe legal and financial consequences on parents who refuse to make their children behave and stop disrupting class. In my opinion, such parental disengagement is educational neglect.

I need some compliant students and parents!!!!

Trackbacks

  1. […] their ratings. The teacher also shares his experience with the new Milestones test.” Article continued… Tell us what you think; add your comments […]