Penn GSE Partnership

Research

The School District of Philadelphia’s school improvement efforts are a central component of its past and current reform efforts and one of its top priorities for research. Shared Solutions’ studies of school improvement serve as the foundation for research that will strengthen the SDP’s school improvement efforts and lead to a rigorous impact evaluation of school improvement efforts in the District.

Our first study uses a matched-comparison, quasi-experimental design to compare the academic achievement of students in a subset of schools undergoing improvement to a comparative group of traditional public schools in the District that were not selected to undergo improvement. Our research plan focuses on answering the following foundational questions:

  1. What are the characteristics of the school improvement models?
    • What are the key characteristics on which the models differ from each other as well as from traditional schools?
    • How do the models evolve over time?
    • What are the supports and challenges to establishing and maintaining the improvement models?
  2. How do student achievement trajectories differ across improvement models, and how do these trajectories compare to those of traditional schools?
  3. What are critical factors in fostering student success in these improvement models?
    • To what extent do policies, school- and classroom-level factors, and context (e.g., socio-economic status, crime rates) mediate or moderate the improvement models’ impacts on student achievement?
  4. Based on our findings, what recommendations can we make to the District for refining school improvement models?

We analyzed data from the 2014-2015 District-wide surveys utilizing an integrative framework that links Bryk and colleagues’ five essential supports for school improvement and Porter and colleagues’ policy attribute theory to explore how the key mechanisms through which these schools’ various reform models work relate directly to student academic achievement as well as how they relate to the existence and quality of the five essential supports—five areas that research has previously identified as indicators of school improvement.

In Spring of 2015, the Shared Solutions team visited 14 schools, interviewing teachers and principals and spending time observing classrooms. This qualitative data contextualized findings from our quantitative analysis and offered hypotheses to explain variation in policy attributes, essential supports, and achievement outcomes.

This important research contributes to the SDP’s goals of improving reform efforts and has already produced instruments and analytic strategies useful to the field. Findings from the study will also facilitate the development of an empirically grounded theory of how implementation relates to effectiveness, paving the way for future reforms that leverage various aspects of successful implementation to influence the essential supports which, in turn, create schools that are successful in terms of academic achievement.