GaDOE responds to Milestones concerns: Test is only one piece of information

A column last week by an Atlanta middle school teacher lamenting the ineffectiveness of the new Georgia Milestones test in helping her improve instruction prompted a lot of discussion on the blog, much of it from teachers with similar concerns.

Among the responses to APS teacher Clara Totenberg Green’s essay was this comment:

DOE says teachers have everything possible “down to the domain level.” That means I now know 40% of my students struggled in the geometry portion/domain of the course; unfortunately there is over 20 standards associated with that domain. I then have to take a stab at which standard(s) needs to be beefed up and wait another year to find out if I made the necessary corrections. To compound the problem not all standards are equal, some are more involved then others.

A teacher who gives a SLO (Student Learning Objective) in contrast will know exactly which domain, which standard, and even the exact question(s) that their students struggled with and can then make an informed decision the following year.  I really would like to hear GaDOE’s response to my post.

Here is the response from DOE Deputy Superintendent for Assessment & Accountability Melissa Fincher:

Domains consist of groupings of similar concepts and skills.  The Georgia Milestones Assessment Guides detail exactly which standards are assessed under each domain.  The domain mastery designation of ‘Remediate Learning,’ ‘Monitor Learning,’ and ‘Accelerate Learning’ provides a gauge of the student’s performance in the domain.

Essentially the domain designation answers the question, “What is the likelihood that the student would be proficient if he/she performed on the overall test as he/she did on the items in the domain?” If it is likely the student would have been proficient, the designation is ‘Accelerate Learning.” If it is possible the student would have been proficient, but equally possible the student might not have been proficient – given the standard error of measurement associated with the domain performance – the designation is ‘Monitor Learning.’ If it is unlikely the student would have been proficient, the designation is ‘Remediate Learning.’

To help educators better understand how students performed on the knowledge, concepts, and skills assessed, Achievement Level Descriptors (ALDs) have been developed by Georgia educators. These are important documents in helping make sense of the test scores and interpret both student strengths and weaknesses as well as reflect upon instructional practices.

The ALDs outline, by standard or group of standards, the knowledge and skills students within each Achievement Level (Beginning, Developing, Proficient, or Distinguished) demonstrate. These documents are designed to show the progression of student understanding and skill for the standard or group of standards described. An example from grade 4 mathematics is provided below. Students who achieve each level demonstrate the knowledge and skills within that level, as well as all previous levels.

Milestones

 

So, if a student or group of students received a domain designation of ‘Remediate Learning’ in the domain of Numbers and Operations in Base 10, the information in the Achievement Levels for Beginning Learner and Developing Learner describe the types of things the student was able to demonstrate – and then those things further outlined under Proficient Learner and Distinguished Learner describe the things the students was unable to demonstrate.

The educational goal is for students to be on track for college and career readiness as they matriculate – and to achieve that level of readiness upon graduation. Proficient Learner is the achievement level that signifies a student is on track. Domain mastery designations in turn, shed light on the student’s readiness within that particular group of standards. The Achievement Level Descriptors are posted on the Georgia Milestones webpage (reached through testing.gadoe.org), along with several other resources for teachers. Unfortunately, it appears many educators are still unaware of these resources despite GaDOE’s repeated attempts to get them into the hands of teachers.

At the end of the day, Georgia Milestones is a summative measure administered as the school year draws to a close.  All test scores require interpretation and should be considered in light of additional information about the student’s achievement.  No test – including Georgia Milestones – is the sum total of all a student has learned and experienced throughout a school year. Nor is a single test a definitive determinant of a teacher’s instructional prowess. Summative tests, such as Georgia Milestones, sample the knowledge, concepts and skills that students are taught each year and are one piece of information.

The Atlanta teacher who wrote the original blog post used a coaching analogy.  No coach waits until the championship game to evaluate his/her players’ readiness for a game; rather they use practices, scrimmages, and regular season games (analogous to classroom tests) to ensure the players are prepared once they reach the championship game. Likewise, all teachers monitor student learning in a variety of ways throughout the course of the school year to ensure students are learning along the way.

 

 

 

Reader Comments 0

27 comments
bu2
bu2

While it is one piece of information, state law says you have to be proficient in English and at least developing in math to be advanced in 3rd and 8th grade.  Its simply not realistic that APS could hold back 67.8% of their students (3rd-8th grade average failing to meet the English standard) or even 36.1% (% failing to meet the easier math standard).  They waived this last year.  I can't see how they can possibly implement this as written.

FlaTony
FlaTony

The bottom line here is that the DOE has affirmed the fact that the Milestones only provides general feedback on student performance. It does not give any diagnostic information which teachers could use to intervene where there are needs.

The Ivory tower has their collective heads in the sand.

class80olddog
class80olddog

They have it backwards - the test should count 50% towards the student's grade and20% to the teacher!

Christie_S
Christie_S

"No test – including Georgia Milestones – is the sum total of all a student has learned and experienced throughout a school year. Nor is a single test a definitive determinant of a teacher’s instructional prowess. "


The results of this test are 50% of my evaluation.  


Even if my all my observations, examples, and portfolio of instructional methods and materials is proficient/exemplary, if my students don't perform well on this test MY overall teacher evaluation is going to be very poor.

Christie_S
Christie_S

So, in other words, the state isn't going to let us know which of the strands the students struggle with.  


Here's the domain for 3rd Grade Life Science and the strands within the standards


S3L1. Students will investigate the habitats of different organisms and the dependence of organisms on their habitat. 


a. Differentiate between habitats of Georgia (mountains, marsh/swamp, coast, Piedmont, Atlantic Ocean) and the organisms that live there. 

b. Identify features of green plants that allow them to live and thrive in different regions of Georgia. 

c. Identify features of animals that allow them to live and thrive in different regions of Georgia. 

d. Explain what will happen to an organism if the habitat is changed. 


S3L2. Students will recognize the effects of pollution and humans on the environment. 

a. Explain the effects of pollution (such as littering) to the habitats of plants and animals. 

b. Identify ways to protect the environment. • Conservation of resources • Recycling of materials 


If students do poorly in this domain, is it because they can't remember which animals are found in which habitat area of Georgia?  Can they remember what animal is typical to which habitat area? Is it because they can't identify the ways that animals and/or plants adapt to their habitats? Is it because they can't explain what happens to organisms if their habitat changes? Or is it that they can't differentiate between the different methods of conservation? Reuse, reduce, recycle...compare and contrast the methods? Pollution's cause and effects? Which pollution does what?


And the biggie, for third grade...can the students read at grade level well enough to understand what the question is asking them?  I can teach the subject areas all day long, differentiating as we go, with plenty of hands-on activities and experiments.  My students demonstrate their knowledge all year with in-class projects, demonstrations, show-as-you-go activities, etc... This year, however, most of them can't read at grade level, so they miss many questions on written tests.  


This year I teach third grade. Next year I might teach kindergarten or fifth.  I may have a class of 20 or, like this year, 26. If I have a third grade class again, I may have a class with the majority who score below benchmark in reading, like this year.   I may have a class with a substantial TAG/high achiever ratio. That's the joy and challenge that comes with being an elementary school teacher.  


I don't think it's too much to ask the state to sort the domains into strands to inform teachers where the problems lie.  They know which questions the students missed, it's a simple sort on a spreadsheet.



MaryElizabethSings
MaryElizabethSings

@Christie_S 

EXCELLENT.  Everyone should read your post, above. 

Just as proficiency in reading is often based on the individual's experiential knowledge of the reading material's content coming into the reading of any material, so the success of students in specific objectives and domains in school will often be the result of the proficiency that the individual students have already mastered coming into learning more within a particular objective or domain.

MaryElizabethSings
MaryElizabethSings

@Christie_S 

In addition, the members of the GaDOE would see greater test score increases if they would put an emphasis in developing stronger reading skills in grades k - 12, in all curriculum areas with the training of teachers in how to teach reading-in-the-content-areas with whole group instruction in their individual curriculum areas.

Also, colleges and universities would fare better in percentages of graduation rates from their institutions after 4, 5, and 6 years, if they would establish vital Reading Centers on their campuses, in which every student - who wishes - could improve his/her reading vocabulary, comprehension, and rate of reading skills, free of cost to the students.

Bitcoined
Bitcoined

The small group of blog malcontents who perpetually naysay any reform offer their teachers' union talking points. But the goal of the Department of Education is clearly to offer parents hope that real reforms are on the way.

Finally.

readcritic
readcritic

@Bitcoined You are expecting a band-aid to stop the bleeding of a severed artery. It is not a small group of teachers pointing out problems with educational reforms. It is a large group trying to show what is not working and they are being ignored or labeled as malcontents.

Bitcoined
Bitcoined

On this blog one sees the same handful naysaying education reform. Day after day after day.

Christie_S
Christie_S

@Bitcoined No, what you see are the same teachers day after day begging for specific information we can use to implement change in our individual classroom instruction.


Generalities aren't good enough.  Tell me in which area I need to improve, which strands need more instruction, and I guarantee you will see improvement.  Don't just tell me I need to improve my Life Science domain scores. That's NOT useful.

taylor48
taylor48

@Bitcoined Because some teachers have given up talking to the same brick walls day after day.  For some reason, teachers can have NO opinion unless it's a union talking point.  (FYI, I don't belong to GAE, so I have no idea what their "talking points" are.  I do know, however, that GaDOE is talking out of both sides of their mouth, when they say that a single test can't show a sum total of what a student has learned over an entire year, yet they use that EXACT SAME TEST to determine whether a teacher has been effective or ineffective over an entire year.  Contradiction much?)

jaggar1
jaggar1

The Georgia education system is so beyond bad that they need to stop. We have kindergarten students chewing on their sleeves all day, because they are so stressed out. Kindergarten should be a time to teach children how to socialize, make friends, and get along in a group. They should learn their A,B,Cs and numbers. No, we have them so stressed out that they cry and melt down. They need to be able to count to 100 by Christmas, add and subtract, read, write paragraphs, and no naps all by Christmas. We need desperately to get back to basics. This ridiculousness of having to read multistep math problems and write about them has to stop. We need to teach them the regular way to add, subtract, multiply and divide. Frankly, these new methods are confusing for them and once they finally understand them, we change it to the old school method. What was the point? It is killing these kids and the teachers. Why do kids need to take tests on science and social studies in kindergarten? They can't recall anything the next week. Focus on reading, math, and social skills the first three years of school. Stop testing these kids to death! It truly doesn't make a difference. Parents-keep your kids home during Milestones and make a stand!

liberal4life
liberal4life

In other words, no specific information that will help teachers improve their practice.

Wascatlady
Wascatlady

It is "one piece" that determines 50% of teacher evaluations?  And yet it does NOT seem to have 50% influence on student expectations for grades, promotion?

Wascatlady
Wascatlady

"Summative tests, such as Georgia Milestones, sample the knowledge, concepts and skills that students are taught each year"


No, those tests are an avenue for students to demonstrate what they have learned, IF THEY CHOOSE TO DEMONSTRATE.  It does not mean they were not instructed, but they may or may not have the motivation to both incorporate the instruction, and to demonstrate mastery when called upon.

CSpinks
CSpinks

@Wascatlady Might students be more likely to choose to demonstrate their skills were so doing required for their promotion to the next grade?

jaggar1
jaggar1

@CSpinks @Wascatlady No. How is it fair to educators who teach the material and a student chooses to not demonstrate they learned the material? The ESOL population may come to school and they may not. They may study and they may not. It is on the teacher when the test scores come out though. Ridiculous!

eulb
eulb

@Wascatlady  Thanks for pointing out that a students' poor grade does not necessarily mean their teacher failed to teach the material.  There's got to be a fairer way to measure teacher performance than this.

readcritic
readcritic

@jaggar1 @CSpinks @Wascatlady The public has no idea how little interest the students demonstrate on tests. Many high schoolers will not even bother to read the questions and will just fill in patterns on answers sheets to "get er done" so they can use their phones or sleep.  Some students are in class only one day a week or one day a month due to incarceration, truancy, pregnancy, etc.  None of these extenuating circumstances is ever considered. The teacher must be effective, engaging, and entertaining at all costs (and the teacher is the only responsible party) with little or no administrative support. Evaluations are wielded against teachers, not to help them improve but to keep them in check.  It is no wonder that teachers are leaving the profession with such odds stacked against them.

readcritic
readcritic

@eulb @Wascatlady Evaluations are too subjective and cost the teacher his/her salary and teaching certificate. Administrators stack the deck against teachers who are not their "yes" followers by giving them large classes of lower level students with numerous behavior issues and no administrative support, and then zap them on the evaluation. The teachers have no recourse.  

Anotherteacher
Anotherteacher


My main frustration with the Milestones test is the lack of practice questions released to teachers.  I teach high school Economics, which has an EOC assessment.  The state DOE has released to us an "assessment guide" which describes the standards addressed and gives a few example test items.  But there are only 15 practice questions in that document.  This is a pathetically paltry amount of practice questions.  There has never been a full-length EOC test released to teachers, and I am not aware of any plans to do so in the future.  By contrast, the previous EOCT assessment did release a full-length exam to teachers to give us an idea of the kinds of questions that would be posed to our students.  Similarly, high-quality national tests (AP, SAT, PSAT, etc.) all have plentiful amounts of released exams that students can use to practice and teachers can use to direct their teaching towards the ideas, skills, and themes that the test-makers emphasize.  To base a large amount of a student's grade (20%) and a significant chunk of a teacher's evaluation score (50%) on a test shrouded in secrecy is problematic. 



I have no problem with my students being evaluated by a standardized test, and I don't object to the idea that part of my evaluation as a teacher should be based on how well I am able to prepare students to take that exam.  But if that is to be the case, I want to be darn sure that the test is decent and that I am given adequate materials and example questions to help hone my teaching and help my students to practice the skills and ideas that they will have to know on the real test.  It would not be difficult for the state DOE to release more practice exams/previously used test questions.  And the cynical part of me wonders why they do not.  Is it because they are too cheap to buy the necessary resources from the test vendor?  Is it because they are incompetent and don't realize the importance of giving teachers adequate preparation materials? Or is it because they are embarrassed by the low quality of the questions on the exams, and are therefore hiding the questions to avoid the outcry that would result if people knew just how poorly-written some of the test items really are?

CSpinks
CSpinks

@Anotherteacher Is my concern that review of readily available practice tests might become the core of classroom instruction unfounded?

Anotherteacher
Anotherteacher

@CSpinks @Anotherteacher  I think you are correct -- if the state releases tests, teachers will guide their teaching towards the topics that the test seems to emphasize.  As long as it is a good test, I don't think this is a problem.  In fact, it is entirely appropriate for teachers to do so.  What gets measured is what gets emphasized.  One of the reasons it is so important for the state to release examples of the new standardized tests is so that everyone will have a chance to examine them to see if the tests are any good or not.  If they turn out to be decently written, great.  I am happy to teach to a well-written test.  If the test really does assess student knowledge about the big themes of my course, if it really is closely aligned to the curriculum guide, and so on, then I have no reason to complain about students having to prove that they have learned that material.  If the standardized tests are badly-written, on the other hand, or if they mostly test knowledge of trivial or unimportant minutiae, then all of us ought to be outraged that they are given such a heavy weight.

Anotherteacher
Anotherteacher


@readcritic @Anotherteacher  I certainly agree that learning is a two-way street, and that both teachers and students shoulder responsibility for making lasting learning happen.  Even the best teachers may not always reach every student, and there are tons of factors outside of the classroom that affect educational outcomes. 

However, it is also true that complaints about bad student behavior are often used by teachers who are simply bad at their jobs.  Why is it that year after year, teachers who teach classes in the same school buildings with similar student characteristics (SES, ability level, etc.) achieve vastly different results as measured by standardized tests?  I'm not talking about outliers or small sample sizes or special situations.  I'm talking about the same school buildings, same type of students, year after year after year.  Why the disparities?  Because some teachers are better at their jobs.  And some are worse.  Everyone knows this is true, and actively works to get their own children into the classes that are taught by teachers with a good reputation.  Why can't we as teachers just admit the self-evident fact: schools are filled with individual instructors who have varying degrees of competence and expertise at their jobs.  Some are great.  Some are okay.  And some need to find other employment.

readcritic
readcritic

@Anotherteacher No teacher, no matter how wonderful, can force students to learn. The teacher, like a doctor, can diagnose and prescribe, but if the student refuses to take the recommended course of action, the cure is ineffective. No one blames the doctor for a patient's bad behavior and failure to follow the plan because the patient has a responsibility to and a stake in his own health. A doctor does not treat up to 40 patients at one time, but teachers are forced to do so daily and then get evaluated on results skewed by many outside factors, while students have no accountability for their own education. Many students choose to commit educational suicide daily, but only the teacher is to blame and responsible for results. Where is the logic?

MaryElizabethSings
MaryElizabethSings

Excellent response by the GaDOE.


I hope that the citizens of Georgia who are not in education will be able to see from both Ms. Totenberg Green's excellent essay and this excellent response to it from Melissa Fincher of the GaDOE, that educators are aware that students are and will be functioning at differing levels on differing skill sets throughout their school careers.  It seems to me from having read both essays that these attempts to individualize instructional delivery is a form of continuous progress to mastery learning.


I simply want to emphasize, having been the ILT in a model continuous progress to mastery learning school for a decade, that students will learn varied concepts at varied RATES, therefore TIME is the fact that must be considered for some students to achieve high school mastery of course work.  I ask the GaDOE to consider incorporating a grade level 13, in the near future, for those particular students to remain in secondary school longer than the norm for their peers in order to achieve mastery of the high school curriculum.