Toward Evidence-Based Education Policy

Basing policy on scientific evidence could transform the field of education — and improve children's learning.

By Marilyn Sherman

When three SESP professors set off for Capitol Hill on a bright May morning, their aim was to convey the latest and strongest evidence in three key areas of education. As the call for — and the controversy over — evidence-based education persists, these prominent national experts brought to light what the best available research could tell policymakers about improving children's achievement.

What is a teacher's impact on student achievement? Which preschool programs improve academic success? How do government policies aimed at families affect children's achievement? These questions were in the spotlight as education researcher Larry Hedges, social psychologist Thomas Cook and economist Greg Duncan addressed top issues at a policy briefing sponsored by Northwestern's Institute for Policy Research (IPR).

SESP professors Larry Hedges, Greg Duncan and Thomas Cook presented the latest and best education research on children's achievement at a policy briefing on May 19 in Washington, D.C. PHOTOS BY LESLIE KOSSOFF/LK PHOTOS
SESP professors Larry Hedges, Greg Duncan and Thomas Cook presented the latest and best education research on children's achievement at a policy briefing on May 19 in Washington, D.C.

The Evidence
Hedges focused on measuring the effect teachers have on student achievement. In his presentation "Teachers: How Much Difference Do They Make and for Whom?" he conveyed the scientific evidence that teachers matter significantly more for student achievement than schools — and teachers matter most in low-income schools.

"It matters more which teacher you get in a school than which school you go to," said Hedges, pointing out key studies. "In addition, teacher effects are bigger in low SES [socioeconomic] schools."

Consequently, Hedges recommended education policies that center on teachers — policies that favor teacher development, teacher accountability and retention of quality teachers in low-income schools. "Incentives to attract and keep effective teachers in poor schools could be particularly effective," he noted.

Cook's "Pre-K Programs: Which Ones Make a Difference?" highlighted research on pre-kindergarten programs. Cook looked at research on different types of public preschools, including Head Start and various state programs. He reviewed what they reveal about cognitive, health and emotional gains.

The evidence shows that universal preschool would improve student achievement. "All the studies show positive cognitive gains," he explained.

However, he cautioned that there is no conclusive evidence about which type of center-based pre-K education works best. The studies are not comparable. Consequently, in response to the debate over federal Head Start programs vs. state block grants, he said, "Policy decisions predicated on state pre-K having bigger effects than Head Start have no solid warrant in science."

Duncan's presentation "Family Economic Policies: Which Ones Raise Children's Achievement?" underscored the importance of looking beyond schools toward economic policies directed at families.

"Policies that increase income bring benefits to younger children," noted Duncan. His examination of welfare experiments and earned income tax credit studies showed that increases in family income bring achievement gains for young children. Significant increases resulted from income gains of only $3,000 a year.

"The design of work-support programs and tax policies can indeed affect the achievement of children," he maintained. "We should really put these kinds of policies alongside of preschool policies and school-based policies in thinking about an array of policy approaches to try to boost kids' achievement."

The Policy Impact
"We disseminated research on children's achievement by three of IPR's and SESP's best researchers in the area to a very wide audience in Washington," IPR director and SESP professor Fay Lomax Cook says of the briefing. She anticipates that it will direct attention toward how to develop more effective teachers, think systematically about pre-K programs, and weigh economic policies such as tax credits and welfare in relation to children's achievement.

From the national level to the local school level, the challenge is deciding how to put education resources to better use. How much will solid evidence guide policy decisions affecting the nation's children?

Not only is there disagreement over what constitutes evidence (see sidebar) but politics also comes into play. "The evidence may not be powerful enough to outweigh the political differences," says Duncan. Furthermore, local control of schools makes it difficult to achieve nationwide initiatives such as national tests or curricula.

However, Duncan notes, "In the case of welfare reform, Congress has learned to value evidence from random assignment studies." And in education the scientific evidence for certain practices is significant, as the recent policy briefing showed. "Our hope is that people will develop an appreciation for telling research and its policy lessons."

In the long run, education may be transformed into an evidence-based field. After all, there may be controversy over what constitutes evidence. However, as Lomax Cook maintains, "I don't see how there can be any controversy over using evidence to shape programs to teach children."

Evidence-Based Policymaking Under Fire

"It's hotly contested," according to Thomas Cook.

"There is a huge controversy," says Greg Duncan.

Evidence-based education policy is on the line. What makes it so controversial?

Duncan says the controversy is over "which methods are most telling for education research." As Cook explains it, "The main question is 'To what extent should knowledge of what works in education be based on evidence-based education research?'"

Many researchers claim that evidence-based education research is the gold standard. To determine what works in education, they favor scientific experiments that randomly assign children to programs and use a comparison group. Progress in education will come only through more scientific evaluation of programs, they say.

In Duncan's view, educators should be able to pick from curricula that have proven themselves. He compares this process to the randomized trials that guide drug approvals. "The country should have evaluation studies conducted with rigorous criteria on a host of curricula," he contends.

Research reactions

At the same time as evidence-based policymaking is gaining fans, there are reasons why some educators have backed off from using scientific experiments to tell them what works. They prefer to rely on experience, theories or the advice of consultants. For philosophical and practical reasons, they frequently avoid random-assignment experiments in schools.

One of the issues is how to measure learning. In fact, some educators maintain that standardized tests do not adequately assess inquiry learning and other important areas. Duncan concurs, "We need to ensure that tests reflect what we want students to learn."

Even though randomized trials offer the strongest scientific evidence for guiding policymakers, SESP researchers see value in other research methods too. "Some people in education … believe that more qualitative small-scale case study research is just as important," explains Fay Lomax Cook.

"The ideal would be experiments or quasi-experiments to understand what works best and smaller qualitative studies to get inside the black box to understand why the programs work the way they do."

By Marilyn Sherman