I was excited to read Thomas J. Kane’s March 5, 2015 piece for the Brown Center on Education Policy at Brookings on how to promote improvement in education. He had my attention from the title onwards, “Frustrated with the pace of progress in education? Invest in better evidence.” As someone who has been making the case for more and better evidence and promoting its widespread use in education for two decades, I welcomed this paper by a leading researcher.
Kane makes four important points regarding why investments in education research and evidence use can promote faster progress in U.S. education reform:
1. "In education as in medicine, most new ideas will fail.” Check! Education is a complex system, and policymakers and practitioners must factor in the expectation of failure. This will help to avoid the “blame game” when things don’t work out as hoped and promote continuous progress based on rigorous evaluations.
2. “Even visionaries need evidence to galvanize others.” Agreed! Kane points out that, “We don’t lack innovation in education. We lack the ability to learn innovation.” Education will benefit when evaluations are an integral part of all improvement initiatives.
3. “Grassroots, small scale trial and effort will never discern the effect sizes we should be expecting.” Definitely! At some point, when an intervention begins to serve many teachers or students in multiple locations it should be tested through a large-scale evaluation that takes into account local variations and other contextual factors.
4. “Federally-funded studies are great for informing federal policy decisions, but we need more investments in evidence by state and local decision-makers.” Excellent! Kane argues that we need “new venues for state and local leaders to make sense of the latest findings.”
Then, at the end of the piece, after I’d agreed with almost everything he’d said, Kane throws in the observation that the “Institute of Education Sciences provides $54 million per year to regional educational labs. Those dollars are not having much impact.” Suddenly, I went from nodding to shaking my head. Where did this unfounded claim that regional labs are ineffective come from? Where is the evidence for this statement? None is provided.
My organization manages Regional Educational Laboratory Northwest, one of the 10 regional educational labs (or RELs) in question. Because of this, you may not trust my word on REL effectiveness. So, let me point you to what some of our state and local leaders have to say about our help in translating research and data into solid improvement efforts.
I point you first to Rich McBride and John Welch, superintendents of Educational Service Districts (ESDs) in Washington state who recently wrote about their partnership with REL Northwest:
Working together in a research-practice partnership, the REL Northwest team gave us deeper knowledge about assessment, how to gather and utilize quality data, and the best ways to report our findings back to the legislature. They helped us think about data in much more effective ways and guided us in asking critical questions . . . There’s nothing like the perspective of an outsider with a deep research base to help you think about how to better operate as a system.
In addition, REL Northwest has supported us in evaluating the quality of the professional development that ESD coordinators in math, science, and English language arts provide to Washington teachers. . . [T]hey’re also assisting us as we collect better data on the coordinators’ impact. They conducted briefings on the best practices of professional development, completed a literature review, and helped us craft research questions (for example, how many hours of professional development does it take to impact teacher practice?). . . .
The study REL Northwest conducted, Coordination of Instructional Services by Washington State’s Educational Service Districts, has received national attention and provoked questions locally about the services we offer. . . . Moving forward, we are using this information to better coordinate services and determine which services to prioritize.
As a network, we are doing the good work we’ve always done but now with a more effective, clear, and aligned systems lens, a research base, and a data-collection process to prove that what we are doing works.
Or, you might wish to read the Education Week Commentary that Beaverton, Oregon Superintendent Jeff Rose penned a year ago regarding the Oregon Leadership Network (OLN) research alliance in which his district participates. Local superintendents were concerned that discipline practices in their districts were having a disproportionate and negative impact on students of color. The districts partnered with REL Northwest to discover the extent of the problem, and then to implement, test, and refine solutions. This is how Rose described the process and its value:
Our research alliance began by trying to assemble a more accurate picture of the extent of the discipline-inequity problem. . . .
To help, REL Northwest prepared three reports to increase each district's knowledge of the issues. First, REL researchers reviewed almost 9,000 articles and prepared a literature summary of school and classroom factors associated with lower rates of discipline. This gave us some concrete strategies to consider adapting to our own context.
Next, REL analyzed the districts' discipline data by race/ethnicity, special education status, and grade level to help each district identify the students who are most frequently and severely disciplined. For me, this clarified how big an issue discipline disparity is in Beaverton's 51 schools. We learned, for instance, that, in February 2013, 7.4 percent of black middle school students in Beaverton had been suspended or expelled, compared with 2.7 percent of white students. In February 2014, the percentages were reduced to 6.5 percent of black students and 2.2 percent of white pupils, but there's more work to do.
In the third report, REL reviewed individual districts' discipline policies and procedures and described how each aligns or does not align with recommended best practices and Oregon school discipline policies. Leaders in Beaverton and the other districts were able to see where we had supportive and equitable policies in place and where gaps existed.
The district partners know that pinpointing the problem is not enough. . . . This partnership work has given us new ideas about how to move forward. This school year, [the partners] are identifying strategies to enact solutions that will work for our schools and students. Alliance members are also identifying common measures that schools might use to pinpoint their needs and track progress.
. . . While we have a long way to go to achieve equity, I now have confidence that principals in my district are increasingly asking the right questions. They are also receiving high-quality data to provide accurate answers about what to do, and how well their actions are affecting results. The support of REL Northwest has been a critical factor in leveraging our progress.
These are but two of many examples that I can cite from REL Northwest’s work to support state and local education leaders in Alaska, Idaho, Montana, Oregon, and Washington. The approach we follow, to establish research practice partnerships that use evidence to foster continuous improvement is inspired by the pioneering work begun in 1990 by the Consortium on Chicago School Research. This promising partnership approach has been well-documented through the support of the William T. Grant Foundation.
I believe that RELs are exactly the right “venues for state and local leaders to make sense of the latest findings” called for by Kane. I encourage everyone interested in the REL system and the work of its 75 research alliances to visit the Institute of Education Science’s REL home page and judge whether the numerous studies and activities seem like they might help education leaders to use evidence to promote improvement.
If Kane wants to promote the cause of evidence use in education, I suggest that instead of criticizing RELs, he devote his next paper to advocating for the ongoing funding of the Investing in Innovation (i3) program. The program is designed in recognition that funding for innovation in education should be based on the evidence of the likelihood that an approach will be successful. Approaches with stronger evidence of effectiveness are eligible for larger awards. All 143 currently funded i3 efforts must be evaluated, so that their promise can be demonstrated. In this way, effective and scalable approaches will be sustained. Currently, Knowledge Alliance is leading a group of nearly 120 organizations that are calling for ongoing funding of the program. I urge Kane and the Brown Center on Education Policy to join the list of signatories.
Steve Fleischman is CEO of Education Northwest and chairperson of the Knowledge Alliance Board of Directors. Knowledge Alliance is a nonprofit, nonpartisan membership organization focused on learning and applying what works to dramatically improve K–12 public education. Knowledge Alliance advocates for the greater use of research-based knowledge in education policy and practice at the federal, state, and local levels. The organization is comprised of leading education organizations that are dedicated to solving some of the biggest problems facing our schools today through the development and use of high-quality, relevant research.