| |
The goal of this project is to design and validate a series of research instruments to identify leadership for mathematics instruction in middle schools and for documenting instructional leadership practice. Adopting a distributed perspective on leadership, this work focuses on both formally designated and informal leaders and their leadership routines.
Our objective is to develop valid and reliable instruments that make the day-to-day practice of school leadership for mathematics instruction more transparent, as well as measure changes in this practice over time. This project utilizes Social Network Surveys, Experience Sampling Methods (ESM) and Daily Practice Logs. To validate these instruments we are using a combination of shadowing, end of day cognitive interviews, and semi-structured interviews. These instruments are used to describe and analyze when and how teachers and other educators solicit or provide instructional advice and the degree to which these resources influence their work.
The following are resources and findings from the Distributed Leadership for Middle School Mathematics Education Project.
Principal Experience Sampling Method (ESM)
The ESM log is designed to determine the activity of the participants at a specific moment in time with a series of signals. Using handheld computers (PDA), participants are beeped at randomly selected times throughout the workday alerting them to fill out a brief questionnaire programmed on the PDA. The questionnaire asks participants to document the type of administrative or curricular activity in which they are currently engaged.
Implementation
The ESM log for school principals was piloted in 52 elementary, middle, and high schools in Spring 2005. Forty-two of the fifty-two school principals completed the log on a regular basis over a 6-day period. At 15 randomly selected times throughout the workday the PDA would beep and vibrate, alerting them that it was time to fill out a brief questionnaire programmed on the PDA. In order to validate the instrument, we shadowed a sub-sample of the school principals and had all 52 school principals complete an end of day (EOD) practice log. We also conducted an email-based survey of the school principals at the end of the logging period that focused on their experience completing the log.
Results/Findings
The majority of the analysis performed on these data has pertained to the validity and reliability of the methodology of ESM. Our findings suggest that there is a high level of agreement between these data and the data that were captured via an end of day web log, specifically in regards to how principals spend their time. The ESM data was also found to be highly correlated with the data logged by a shadower who observed five different principals for one day each. Our findings suggest that ESM is a valid and reliable method for collecting data. This is a significant finding in that ESM drastically reduces recall bias while maintaining reasonably high response rates.
In an attempt to determine instrument reliability and validity, we compared the results of the ESM log with an end of day (EOD) practice log that also captures how principals spend their time. A significant correlation was identified when comparing the percentage of time spent each day on Administration, Instruction and Curriculum, Professional Growth, and Fostering Relationships that was captured via the EOD and ESM logs. In both instruments, the percentage of time spent on administration and instruction or curriculum exceeded 80% of the total time reported. Using regression techniques, we confirmed a high-level of agreement in these percentages. To account for inflated agreement due to other factors, the regression models that were used to calculate the correlation coefficients controlled for day, time, non-response, and principal effects.
As an additional means of validation we shadowed 5 principals and observed them each for 1 day. This data was then compared to the ESM data and correlation coefficients were calculated between the ESM and the shadower data on the same measure. Results indicated that the relationship between these values was significantly correlated.
School Staff Network Survey
The social network survey is designed to focus on mathematics instruction for the middle school grades (grades 6-8) and to investigate the sources of instructional leadership that influence teacher instructional practice. The survey is administered and completed online and takes approximately 30 minutes to complete. Questions pertain to activities and opinions related to school leadership in instructional improvement, expertise in mathematics leadership and the conditions associated with each. The survey specifically asks participants to indicate the people, resources, and activities they seek out to gain information and advice related to teaching in general, math, and literacy.
Phase I Implementation
The school staff network survey was piloted by 90 teachers and administrators at two middle schools in a suburban school district. We also conducted interviews with a sub-sample of leaders and teachers in both schools.
Phase II Implementation
The school staff network survey was revised based on our analysis of the pilot data and implemented in 21 urban schools, where 362 teachers and administrators completed the survey.
Results/Findings
The survey results indicate that participants most frequently sought advice or information from all roles on a weekly and monthly basis. While advice or information was most sought from principals and assistant principals on a monthly basis, the advice of math and reading specialists was most frequently sought on a weekly basis. It is interesting to note that math and/or reading specialists were sought out twice as frequently as principals and assistant principals.
Similar to the results for people, the results indicate that respondents most frequently engaged resources or activities on a weekly or monthly basis. The only notable exception is the use of textbooks, with their use being on a daily basis.
Validation of this instrument is ongoing and will consist of a review of interview data to confirm the results of the survey.
Instructional Leadership Daily Practice Log
The daily practice log is designed to document some of the interactions instructional leaders have with their colleagues. Both administrators and teachers are asked to complete the log to capture the nature of the interactions from the perspective of formal leaders and non-leaders. Each day participants are asked to log one interaction per hour that they had with a colleague, group or resource that influenced their knowledge, practice or motivation. Participants are also asked to note what prompted the interaction, who was involved, how it took place, what transpired, and the academic area it pertained to.
Implementation
Thirty-seven school leaders and teachers from a sub-set of schools that completed the school staff network survey were asked to complete a daily practice log to capture the nature of their interactions pertaining to instructional leadership over a 2-week period.
Results/Findings
The results of the daily practice log indicate that only a very limited number of interactions related to mathematics take place on a daily basis. The log data suggests that when school personnel interact with a colleague it is most often in regards to disciplinary and administrative issues and rarely related to math or instructional issues.
Validation of this instrument is ongoing and will consist of two stages: 1). The review of shadowing data from 20 log participants who were shadowed by researchers over a 2-day period during the log completion period. During this time researchers documented all interactions that had the potential to impact the participants’ knowledge, practice or motivation; and 2). Interview data will also be used to confirm the results of the log.
Assessing the Import of Principals' Professional Development: An Evaluation of NISL
The primary objective of the NISL project is to assess the effects of principal participation in the National Institute for School Leadership professional development program.
The evaluation will focus on area that are thought to support instructional improvement, particularly connections between principals’ practices, improvements in teachers’ practices, and gains in student achievement. The study is directed by the Consortium for Policy Research in Education (CPRE). The Consortium, based at the University of Pennsylvania, unites researchers from five of the nation’s leading research institutions in an effort to improve elementary and secondary education through practical research. The CPRE researchers working on this evaluation are James Spillane of Northwestern University, Carol Barnes and Eric Camburn of the University of Michigan, Susan Fuhrman and Jonathan Supovitz of the University Pennsylvania, and Ellen Goldring of Vanderbilt University.
CPRE’s evaluation features a mixed-method research design which will allow us to explore fine-grained issues of principal’s knowledge base and instructional practices while examining these practices and their links to teaching and learning on a large scale. A central feature of the approach is a randomized, delayed-treatment design that compares elementary school principals from an urban school district who are randomly assigned to participate in NISL in the first year of the study (early-treatment group) with principals from the same school district who are randomly assigned to a group that receives the treatment one year later (delayed-treatment group). We will work closely with NISL and school district staff to ensure that the assignment of principals to early and delayed-treatment groups is done completely at random. About 20 principals will be assigned to each group. CPRE will follow these principals for three school years. Randomized assignment will permit us to make strong inferences about the effect of NISL participation on principals’ knowledge and practice. This approach is designed to allow us to go beyond the assessment of main program effects and develop hypotheses about how principal development programs can affect school outcome more generally. The design will also include surveys and qualitative data collection that are intended to complement the randomized trial by supporting the identification of factors that may explain the effectiveness or ineffectiveness of NISL.
The Educational Leadership Collaboratory builds on the work of Jim Spillane, and introduces school leaders to the distributed leadership perspective, identifies ways to put it into practice and identifies ways to use this knowledge to create a community of practice that will foster improved teaching and learning.
The object of the Distributed Leadership Collaboratory is to develop three foundational learning modules within the executive education program of our Educational Leadership Collaboratory. This project will fully develop our preliminary design work through a process of pilot implementation, evaluation and refinement, implementation, and curriculum development and dissemination on three topics: The project introduces school leaders to the distributed leadership perspective, identifies ways to put it into practice and how to use this knowledge to create a community of practice that will foster improved teaching and learning. These modules will then be made available for use by school districts, university faculty, and other agencies to improve leadership practice.
The core challenge facing the American education system is the improvement of teaching and learning in urban schools. For too long, poor students and students of color have been denied access to quality opportunities to learn. Meeting this challenge will depend on a substantial improvement in the quality of instruction in America’s schools, especially urban schools. Instructional improvement depends on school-level factors, especially school leadership (Leithwood, et. al, 1984; Barth 1986; Blase & Blase, 1999; Marks & Printy, 2003). Schools that create incentives and opportunities for improving teachers’ practice tend to have shared visions about instruction, norms of collaboration, collective responsibility for students' academic success, and an ongoing reflective dialogue among staff about practice (Bryk & Driscoll, 1985; Newman & Wehlage, 1995). School leadership is essential for the development and maintenance of these conditions. Unfortunately, most school leaders are ill prepared to create and sustain these conditions. Improving leadership development is essential to significantly improve learning opportunities in urban schools.
The focus of the Searle study is to examine the ways teachers’ motivation, support and knowledge are related to the type and organization of the school.
The Distributed Leadership for Elementary School Leadership study was a longitudinal study of urban school leadership funded by the National Science Foundation and by the Spencer Foundation in collaboration with people from the School of Education and Social Policy at Northwestern University.
Building on theories of distributed cognition, this program of research was designed to examine the practice of school leadership in urban elementary schools that are working to improve mathematics, science, and literacy instruction. There is convincing empirical evidence to suggest that the school (not the district or the state) is the most important unit when it comes to the enactment of instructional improvement. This previous research makes the case that instructional improvement depends on school-level factors, especially the school leadership provided by the principal, other administrators, and teachers. However, while we know a considerable amount about the organizational structures and leadership roles that contribute to improve instruction, we know relatively little about school leadership as a practice. Further, what we know has focused almost exclusively on the school principal with limited attention given to other administrators and teachers who frequently take on leadership responsibilities. Finally, the connections between leadership practice and teaching practice are not well understood.
This study was designed to explore and understand leadership as a practice of instructional improvement and to examine the relations between leadership practice and teachers' classroom work. Our goal was to construct a theoretical framework that is grounded in the day-to-day practice of leadership, thereby increasing the relevance of this conceptual framework to practice. We believe that if theory is to be more influential in practice, it will need to grow from practice, providing a frame that helps practitioners interpret and reflect on their ongoing work. Our plan is to share our research findings with policy makers and educators in other schools in order to help them think about their leadership practices.
Last Updated: 2006-02-14 12:29:05
|
|
