SESP Researchers Use New Tools for Understanding Education Policy

“We’re starting to create and refine a general methodology of treating education policy issues using complex systems perspectives. By taking the two most prominent tools, agent-based modeling and network analysis, we’re able to make progress on difficult and thorny policy issues,” notes SESP and McCormick School of Engineering and Applied Science professor Uri Wilensky, an author of the Science article along with Luis Amaral of McCormick. “Once we make simulations, we can explore alternatives and see the outcomes.”

“This is a breakthrough in the sense that this lens that has been used inside classrooms to help students learn is only now being turned on the education system as a whole,” says Spiro Maroulis (PhD08), Wilensky’s former graduate student and now a visiting professor and postdoctoral fellow at Northwestern’s Kellogg School of Management. Wilensky, Maroulis, Amaral, SESP doctoral student Hisham Petry, former SESP faculty member Louis Gomez and Northwestern researchers Roger Guimera and Michael Stringer describe an innovative approach to education policy research that can provide new understanding of how school reforms happen.

For about seven years, Wilensky has been using this new method of applying complex systems to education. This type of research has the potential to increase understanding of education in a way that can help policy makers to bring about improvement and change.

Agent-based modeling is a type of computer simulation that is used to examine how the behavior of systems with many interacting individuals unfolds over time, and social network analysis frames and quantifies the social interactions between people in such systems. These tools allow the researchers to model schools at the level of individuals and see the outcomes that emerge from the choices and behaviors of individuals and their interactions. For example, new tools for visualizing social networks show how specific classroom interactions such as the content of student conversations lead to outcomes such as classroom order.

In the past, the two basic categories of education research — “effects-based” quantitative research and “mechanism-based” individual-level research — never were clearly connected, according to Wilensky. “What we’re attempting to do is connect up the micro with the macro.”

“Agent-based modeling and network analysis can help integrate knowledge on ‘what works’ in education with knowledge on ‘how it works,’” Maroulis adds. “Education researchers know a lot about what’s happening on the ground in schools, and we can also measure and compare the outcomes that come from that activity across schools,” he explains. “Agent-based modeling allows us to run lots of different scenarios and investigate which ‘on-the-ground’ scenarios are most likely to yield the outcomes we observe in effects-based research, or even better, we can run those scenarios before we design and execute our expensive real-world experiments.”

The researchers have used a modeling approach for a number of topics, including the hot educational issue of school choice. Using actual data from Chicago Public Schools and Wilensky’s NetLogo modeling software to simulate school choice scenarios, the researchers gained some novel insights. Surprisingly, they found that in a choice environment often it wasn’t the case that the best schools survived, according to Wilensky. Maroulis explains further that that the more people who participate in a school choice program, the more misleading the comparisons of the academic outcomes between those who attend “choice” and neighborhood schools become.

In addition, school choice results differed based on what students and their families valued. When the researchers varied how much value was placed on academic achievement rather than geographic closeness, they found that valuing achievement highly actually limited how much achievement could increase in the district. In that case, students were opting not to take a risk on new schools with the potential to increase achievement in the district, according to Maroulis. Future models could be used to design school choice program to increase district achievement, the authors contend.

The research produced new findings related to friendship networks as well. For example, by taking social network surveys within high schools and analyzing the subgroups of friendships, studies by Wilensky, Gomez and Amaral showed the impact of peers on achievement. “If you were a low achiever and had friends who were high achievers, your scores improved,” Wilensky notes. Conversely, for a high achiever in a low network, scores didn’t improve. In addition, the gains made by a high-achieving friendship network were higher than those for a low-achieving network, according to Wilensky.

The Northwestern researchers have done numerous studies to explore education from a complex systems perspective, and the results are being published in various journals. Complex systems work on education research has been supported by the National Science Foundation, Searle Foundation and Northwestern Institute of Complex Systems.

At the School of Education and Social Policy, work continues to apply the complex systems lens to education as a way to tackle various policy issues. For example, Wilensky is working with professor David Figlio to study the effects of tracking on student achievement. He sees this type of research, which is new to the field of education, becoming a trend at SESP. “Not a lot of people are using these kinds of methods,” says Wilensky, who predicts that the complex systems lens will become more prevalent.

Read the Northwestern University press release and the McCormick School of Engineering and Applied Science report

Caption:
A computer model visualizes the results of a school choice study, with the large blue circles representing expected enrollment in schools.