TIME Ideas obtained this Pearson memo from an employee in New York state government who is frustrated with the lack of transparency surrounding the recent firestorm over standardized testing. The full text is below, with the exception of a phone number in the last paragraph that has been redacted. To read Andrew J. Rotherham’s anatomy of the scandal, click here.
April 22, 2012
Mr. Ken Slentz
Deputy Commissioner, Office of P-12 Education
New York State Education Department
89 Washington Avenue
Albany, New York 12234
Pearson is confident that the NYS Grades 3-8 English Language Arts (ELA) and Mathematics assessments have been developed to support valid and reliable interpretations of scores for their intended uses. The “Hare and the Pineapple” passage and associated items were placed on the Grade 8 ELA test after the NYSfield test data associated with the multiple choice items and the feedback from the “final eyes” committee determined that this was an appropriate passage and set of items to include on the test. Detailed background information about the passage and items are provided below.
Background on SAT 10 Items and Use in New York State
When the contract was awarded to Pearson in March 2011, part of the scope of work was to include norm-referenced items that would be administered each spring in the New York State
Grades 3-8 English Language Arts (ELA) and Mathematics assessments. These items would serve two purposes – to provide national normative data and to contribute to the student’s operational score. Form B of SAT 10 was planned to be used intact to meet both requirements. Likewise, due to the planned inclusion of these normed items, Pearson planned to meet the item development target numbers with a combination of both normed and custom developed items.
In fall of 2011, the New York State Education Department (NYSED)made a determination that the SAT 10 Form B would not be used in total on the 2012 operational assessment. This decision was made due to the fact that not all SAT 10 items are aligned to 2005 New York State standards and having such items contribute to an operational score was not ideal. With this decision, two shifts resulted. The first is that if any SAT 10 Form B items were used on the operational assessment, they would not yield normative data (as the complete SAT 10 Form B is needed to establish this). Secondly, it was determined that custom developed passages and items should be placed on the operational test forms first, and if there weren’t enough eligible custom items, to use the field tested SAT 10 Form B items.
Why the “Hare and Pineapple” Passage was Chosen
During test construction it was determined that with the exclusion of the SAT 10 items on the operational form there were not enough custom items developed to assess Strand 2, therefore
“The Hare and the Pineapple” passage and associated items were chosen for the operational form. This was a sound decision in that “The Hare and the Pineapple” and associated items had been field tested in New York State, yielded appropriate statistics for inclusion, and it was aligned to the appropriate NYS Standard.
“The Hare and the Pineapple” passage is intended to measure NYS Standard “interpretation of character traits, motivations, and behavior” and “eliciting supporting detail”. The associated six multiple choice items are aligned to the NYS Reading Standards, specifically to Strand 2. The NYS performance indicator assigned to the items is “Interpret characters, plot, setting, theme, and dialogue, using evidence from the text”.
It is important to note that the use of SAT 10 items as operational items will not occur going forward as Pearson is developing an adequate number of custom items aligned to the Common Core Standards.
Concerns with Items Associated with “Hare and Pineapple”
There have been two items of the set of six that have been challenged by NY teachers and students as the test was under way April 17-19, 2012 -Item 7 and Item 8. The correct answers and rationales to Item 7 and Item 8 are as follows:
• Item 7: The correct answer is C. The question regarding the animals’ possible motivation for eating the pineapple requires a reader to infer the correct answer from clues conveyed in the text. While all of the options are plausible motivations, the most likely answer is that the animals were annoyed. Paragraph 13 indicates that the animals support the pineapple to win the race because they assume the pineapple has a clever plan. However, the pineapple never moves during the race. From these clues and events, a reader can infer that the animals are annoyed. The text does not support the inference that the animals are motivated by hunger, excitement, or amusement.
• Item 8: The correct answer is D. The question regarding the wisest animal requires the reader to apply close analytic reading skills to determine which of the choices represents the wisest animal based on clues given in the text. The moose and the crow are the two animals that present the incorrect idea that the pineapple has a clever plan to win the race. This idea is proven false when the hare wins the race. The hare is presented as incredulous that a pineapple would challenge him to a race, but overconfidently agrees to race a pineapple.
Finally, the owl declares that “Pineapples don’t have sleeves,” which is a factually accurate statement. This statement is also presented as the moral of the story, allowing a careful reader to infer that the owl is the wisest animal.