Resources from a “Bridge Event” designed to connect researchers, policymakers, and practitioners—held on August 12, 2010, in Eugene, Oregon.

Resources

• Agenda
• Proceedings
• Handout

Dr. Andrew Ho, assistant professor at the Harvard University College of Education, provided an overview of the problem with proficiency rates as an indicator of education achievement trends, gaps, and gap trends. A follow-up session delved deeper into improved test score reporting using simple data displays, table discussions, and a question and answer period. Ho concluded by introducing a toolbox for test score reporting with advanced data displays.

Keynote Speech

Ho provided context for his research by discussing how the reporting of student proficiency percentage scores (the proportion of students scoring above a preset cut point) is ubiquitous under current state and federal accountability systems and carries high-stakes consequences. Ho demonstrated that measuring progress only by those students who cross a certain cut point yields a skewed and often misleading picture of differences among groups and of educational improvement over time. Such metrics are especially problematic as indicators of progress toward equity in educational opportunity. While Ho does not oppose standards, his research has led him to propose alternatives to the use of proficiency rates for comparing the achievement of different groups or for measuring progress toward achievement of standards.

Improved Test Score Reporting Using Simple Data Displays

Ho reviewed his thesis with a series of PowerPoint graphs. He then asked participants to take a short “quiz” about ways to represent data and asked a few audience members to share their results with the whole group. The exercise made it clear that participants understood Ho’s basic points about how proficiency rates can be misleading.

Ho’s presentation also introduced the primary growth models currently being implemented in various states. He showed how these models are also susceptible to the problems of student proficiency ratings. Then, Ho used student data from the state of Oregon to present simple methods for displaying information in ways that improved on simple proficiency rates. The pros and cons of these graphical presentations were discussed by table groups.

Toolbox for Test Score Reporting with Advanced Data Displays

In addition, Ho shared improved ways to display data. He introduced a “toolbox” of more sophisticated data displays that use distribution-wide percentile displays and effect sizes. Ho incorporated actual Oregon student achievement data into his presentation to demonstrate how the data can tell different stories depending on where the cut point is placed.

Also available is a video on this topic as it was presented in Seattle, Washington, on March 25, 2009.

Attachment
Agenda
Handout
Proceedings