To School Committe Regarding Hand Scoring of MEA

Dear School Committee Members,

By now, it is no surprise to you that I am writing about standardized testing.  Over the last several days, I have gained some knowledge I feel you must also know. 

This ad on Craigslist was circling on Facebook.  The employer is Measurement Inc and they are seeking scorers for open responses on standardized tests.  I even mentioned this ad causally in my public comment on Feb 23rd.

The idea that a Maine high stakes test would be scored by just anyone with any sort of four year degree willing to work for $11 an hour in Detroit is absurd to me.  But, so was a four year old taking a standardized test, and I was wrong about that. 

So, I tweeted the ad:

 jaclyn boyd ‏@jackybboyd

.@LewistonSuper @mdoeNews Is this how our MEA scorers are hired? Make info on this topic public & easily accessible.

I received this as a response the following day:
Maine DOE ‏@mdoeNews Mar 1

@jackybboyd No. Computerized scoring for many questions and open-ended responses scored by educators. Test made by teachers, for students.

I was satisfied for the time being as I had other concerns I was attending to, but a Montello teacher, Emily Kennedy Talmage, also saw the ad.  We started discussing it.  She was particularly concerned that her students, who were working so hard to make gains in their writing this year, would not have their work scored by the most qualified scorers.  This concern for her students spurred her to do some research. 

First, Emily found an article in Ed Week that connects Smarter Balanced with Measurement Incorporated, the company in the ad. 

"McGraw-Hill Education CTB, for instance, told Education Week it is relying on seven subcontractors to help with a $53.7 million Smarter Balanced contract that includes test-item development, review of items, scoring, checking alignment with standards, and project management. The subcontractors include not only individuals providing consulting, but companies and organizations such as the AIR; the Data Recognition Corp., of Maple Grove, Minn.; and Measurement Incorporated, of Durham, N.C." 

Next, Emily found the following on the Maine DOE's web site. 

Who is handscoring?

Individual school staff are being trained to hand-score that component of the interims. The hand-scoring for the summative is being performed by AIR’s sub-vendor, Measurement Incorporated.

Suddenly, my concerns were more than I could express in a tweet.  Even though the Maine DOE had said "no," our tests are not scored like this, that didn't seem to be the case.  I emailed both of the texts presented above to Samantha Warren.  She was out of the office, but responded the following day with this:

"Now that I am back in the office, I wanted to follow-up with you on how constructed responses to the State assessment, which was developed by teachers including dozens from Maine, are scored. As in the past, answers are scored anonymously by educated (four-year degree or higher) specially trained and monitored raters, which often include many educators (a solicitation targeting Maine teachers will go out soon I am told).

I don’t want to debate the merits of posting jobs on one site or another (I know for hiring I’ve done before, I post any place I can that may attract applicants, especially if it is free but of course only hire anyone who is qualified). I know there is no information I can provide you that will increase your confidence in this assessment, and I completely respect that and as I’ve said before, am appreciative of your engagement in improving education on behalf of your kids. All that I can do is provide you timely answers to the many questions you pose."

While Ms Warren's response was certainly respectful, it did not change the following:

1. The initial response from the Maine DOE on twitter and this email do not match.  That calls into question for me just how valid the assertion that teachers were meaningfully involved in the writing of the exam as well as that it will be scored in Maine. 
2. How can we let non-educators be the work force scoring these exams?  Not only does that seems disrespectful to the hard work of students, but it is also disrespectful to the hard work of teachers and the schools as they will also be judged on these scores. 
3. Ms Warren says that this is how the exams have been scored previously.  However, I saw a Maine teacher comment online that when she scored MEA's it was a three day weekend at USM and the scorers were all teachers.  Also, the scorers were not paid, but received professional development credits.  That is quite a different picture to me. 
4. Ms Warren did not provide any documentation showing that Maine tests are not shipped off to Detroit to be scored by the very people hired by this ad. 
5. Craigslist is not the proper venue for posting a professional job such as this, in my opinion.  Teaching jobs in Maine aren't posted there but on Serving Schools; why would scoring be any different? 

I may sound elitist in my desire that only educators score these exams, but I remember what it was like during my student teaching and first year of teaching.  I remember what it was like to learn how to score writing samples and how long it took me to be good at it.  I have witnessed teachers not accustomed to large volumes of student writing struggle with it as I did.  That is NOT the way we want our students' tests scored.  And it is certainly NOT how we want the data generated for our teachers' evaluations and our schools rankings. 

This finding is just another reason these tests are not right for our students.  Best practices are being ignored by politicians and corporations making these decisions. 

Sincerely,
Jacky Boyd