Leonie on Teachers to Be Measured Based on Students' Standardized Test Scores

Click on image to enlarge


"Using a complicated statistical formula, the report computes a "predicted gain" for each teacher's class, then compares it to the students' actual improvements on the test. The result is a snapshot analysis of how much the teacher contributed to student growth. "

Leonie Haimson writes to her listserve:

What factor did they use in terms of improvements -- one year's gains or losses in test scores? Such a small number of students as are included one class would likely lead to an even more unreliable measurement than the progress category at the school level, which culminated in the highly unreliable school grades.

How does such a highly erratic and variable measure get teachers "comfortable with the data, in a positive, affirming way," as Chris Cerf asserts? How exactly does it "help teachers identify their strengths and weaknesses" as Randi writes?

Moreover, according to the "performance predictor" chart above -- the formula was supposed to control for class size at the classroom and school level. Did it?

It appears so. "The teacher data report balances the progress students make on state tests and their absences with factors that include whether they receive special-education services or qualify for free lunch, as well as the size, race and gender breakdown of the teacher's class."

In an oped about evaluating teacher performance in the Daily news in April, Klein wrote that “Nor should test scores be used without controlling for things like where students start academically, class size and demographics.”

http://www.nydailynews.com/opinions/2008/04/08/2008-04-08_beware_the_teacher_tenure_trap.html

Will we ever get to see the formula? How much of a factor did they attribute to class size?

I'd like Eduwonkette and other statistical experts to be able to analyze it.

Sample Teacher Data Reports on Student Test Scores

Usually I'd put these up on Norms Notes. But this issue is important enough to put them on the main blog.

How about that formula for measuring teachers (see below) or analyzed it in terms of its reliability? If it turns out to be unreliable – as has every other formula the DOE has developed in recent years -- how can it be useful even for diagnostic purposes?

The NY Times report today said this:

The teacher data report balances the progress students make on state tests and their absences with factors that include whether they receive special-education services or qualify for free lunch, as well as the size, race and gender breakdown of the teacher’s class.

Using a complicated statistical formula, the report computes a “predicted gain” for each teacher’s class, then compares it to the students’ actual improvements on the test. The result is a snapshot analysis of how much the teacher contributed to student growth.

The reports classify each teacher as average, above average or below average in effectiveness with different categories of students, like those who score in the top third or the lowest third on the test, and those still learning English or enrolled in special-education programs. It also contains separate measurements on effectiveness in teaching boys and girls, though it does not distinguish performance by students’ race or income level. Teachers will also be given a percentile ranking indicating how their performance compares to those who teach similar students and to a citywide pool.

Teaching Resources
http://schools.nyc.gov/Teachers/Resources/teacherdatainitiative.htm

Teacher Data Initiative

A few things to keep in mind when viewing the sample Teacher Data Report:

* The sample Teacher Data Report contains illustrative, not real, data.
* The name of the teacher and the school are fictitious—any resemblance to the name of an actual teacher or school is purely coincidental.
* The sample report is a working draft. The reports' format may be revised based on additional feedback from teachers and school leaders before they are distributed.

Assistance understanding the reports
Schools will receive training on how to read and interpret Teacher Data Reports before they receive their reports. In addition, Web-based tools will be available to help teachers and school leaders understand Teacher Data Reports at the time that the reports are made available.

Click to enlarge