Building Quality in Washington State

The origins of quality improvement

Since 2008, School’s Out Washington has been leading the effort to build and sustain a quality improvement system for expanded learning programs. The impetus for this work is grounded in the belief that all children deserve the opportunity to thrive and an understanding of how historical inequities continue to limit opportunity for children experiencing poverty and racial oppression. 

The reality is that schools cannot meet these challenges on their own. We need an integrated approach to learning across the day and year to narrow opportunity gaps and amplify classroom learning. We know that children and youth who participate in afterschool and summer expanded learning programs can achieve positive social, emotional, and academic gains.^[1] For expanded learning programs, quality of instruction is a significant factor in promoting positive outcomes. High-quality programs are those that provide:

• Physical and emotional safety
• Supportive relationships with adults and peers
• Opportunities for meaningful interaction and leadership
• Highly engaging experiences that enable skill building

Research suggests that programs that reach higher levels of quality, especially in the areas of interaction and engagement, can have significant positive effects on youth outcomes.^[2]

In 2009, the Raikes Foundation recognized the importance of high-quality expanded learning programs and partnered with School’s Out Washington to launch their Youth Program Quality Strategy in Washington State. The strategy engaged an initial cohort of nine youth programs in a quality intervention process developed by the David P. Weikart Center for Youth Program Quality. The Youth Program Quality Intervention (YPQI) pairs quality assessments using research-based, validated tools with targeted training and coaching supports.

Quality improvement gains momentum

Each program that participated in School’s Out Washington’s initial quality improvement cohort received funding from the Raikes Foundation for a three-year intervention cycle. The Foundation then supported the launch of additional cohorts in 2010 and 2011; by 2015 Raikes had supported more than 70 program sites in nine cohorts across the state. In order to effectively scale the YPQI process, School’s Out Washington established “hubs” in King, Spokane, and Pierce counties in 2011. In each of these counties, a hub lead provides local coordination and serves as a point of contact for participating programs. School’s Out Washington continues to provide statewide system-level support in the form of hub calendars and materials and maintains a statewide cadre of coaches and external program assessors.

Using this “hub” regional structure, School’s Out Washington has helped facilitate quality training and evaluation for more than 756 youth programs. And, in addition to the Raikes Foundation cohorts, School’s Out Washington has supported multiple organizations and networks in their continuous quality improvement efforts, including:

• 21st-Century Community Learning Centers (CCLCs)
• Expanded Learning Opportunities Quality Initiative participants
• Best Starts for Kids – King County grantees
• Boys & Girls Clubs of Washington
• YMCA of Snohomish County
• City of Seattle – Dept of Education & Early Learning providers

School’s Out Washington aims to expand the reach of its quality improvement efforts to more programs across the state in the coming years.

Defining standards and core competencies

As School’s Out Washington was working to establish Washington’s quality improvement system, it became clear that the provider community was ready to embrace an aspirational, field-level definition of quality. School’s Out Washington worked with partners to create the Washington State Quality Standards for Afterschool and Youth Development Programs. The standards define what quality looks like in a program setting and apply to all programs for youth ages five to young adult across our state. Based on national research, the standards serve as benchmarks that help programs set and reach improvement goals.

In addition to the standards, School’s Out Washington established the Washington State Core Competencies for Child and Youth Development Professionals. The core competencies define what professionals need to know and be able to do in order to provide quality programming. In order to increase adoption of the standards and core competencies at the program level, School’s Out Washington has developed aligned training modules in structural racism and cultural responsiveness for direct service providers.

Together, these pieces establish a shared understanding of quality that supports the professionalization of the child and youth development field. Ultimately, we hope that this will improve and enrich the experiences that Washington’s young people have outside the classroom.

Current focus areas

School’s Out Washington’s current work focuses on establishing a shared vision for high-quality, equitable, and accessible expanded learning for all children and youth across Washington State. We are actively building the infrastructure and public will to support this work. Current highlights include:

Expanded Learning Opportunity Quality Initiative (ELOQI)

The Expanded Learning Opportunity Quality Initiative (ELOQI) is a statewide initiative that focuses on engaging youth programs in a continuous quality improvement process using the Social and Emotional Learning Program Quality Assessment developed by the David P. Weikart Center for Youth Program Quality. For the past three years, the ELOQI has demonstrated that expanded learning programs, with proper supports, can deliver the high-quality engagement proven to result in meaningful youth-level outcomes. The Washington State Legislature has provided ongoing funding to continue this work.

The Quality Seal

An early goal of the ELOQI was to develop and validate a “Quality Seal.” The Quality Seal is an observation-based assessment tool that complements the PQA and serves as an indication that programs have attained a threshold of quality that is commensurate with achieving youth outcomes. Programs participating in the ELOQI also receive a Quality Seal Assessment from Cultivate Learning at the University of Washington. The Quality Seal recognizes high-quality ELO programs that demonstrate best practices along five dimensions of program quality:

• Relationships
• Programs and activities
• Social and emotional learning
• Assessment, planning, and organizational structure
• Family, school, and community connections

System-level investments in quality

As we work to build the necessary infrastructure to scale quality work across the state, we are encouraged by the promotion of continuous quality improvement across many major systems that serve young people. In King County, all Best Starts for Kids Out-of-School-Time grantees participate in the Youth Program Quality Initiative (YPQI). Washington’s Office of the Superintendent of Public Instruction (OSPI) supports quality work through its investment in 21^st-Century Community Learning Centers. Other major systems that have committed to quality include the Associated Recreation Council of the Seattle Parks and Recreation Department, Boys and Girls Clubs of King County, the City of Seattle’s Department of Education and Early Learning, and the YMCA of Snohomish County

What we’re learning

We know that quality matters. Several recent initiatives have provided School’s Out Washington with opportunities to ask big questions about the impact of quality improvement work in Washington State. Specifically:

• Can high-quality programs affect academic outcomes?
• What elements of the YPQI have the biggest impact on program quality?
• What do program providers most need in order to sustain quality work?
• How can we use data to inform our approach and establish a shared view of our youth program landscape as it relates to quality?

From quality to outcomes

A study conducted in two subsequent years looked at the relationship between responsive, high-quality instruction and students’ academic gains in Seattle Public Schools summer programs. School’s Out Washington provided training and assessment for programs, which combined academic instruction with enrichment. The quality of instruction across program sites was high relative to that seen in other cities, as was student attendance in the programs. Most students in the program improved in both literacy and math as a result of their participation.^[3]

A recent evaluation of Washington’s 21^st-Century Community Learning Centers showed that programs engaged in quality work had a positive impact on students and families. Students who attended these programs regularly had better grades, fewer school-day absences, and fewer disciplinary referrals than similar students who did not attend programs. Students also reported that they had better peer relationships and improved attitudes as a result of their experiences in programs. These findings were echoed by families, who both appreciated the ways that programs benefited their children and felt that they themselves were connected to and supported by the programs.

The impact of coaching

A rigorous evaluation of the ELOQI revealed that coaching helps programs to increase the quality of their offerings. Programs in the study that were initially rated as lower than average in quality improved significantly after training and coaching. Providers in these programs reported that they benefited from coaching, as it helped them to translate quality assessment and training content to on-the-ground practice.^[4] In the second year of the ELOQI, virtual coaching yielded the most significant gains in program quality.^[5] This finding is encouraging, as the virtual format provides an opportunity to increase coaching supports to providers in underserved areas of the state.

Provider perspectives

Much of School’s Out Washington’s work over the years has generated insight into the perspectives of the providers in the expanded learning field. Recent evaluations have confirmed many of these insights. One key outcome of School’s Out Washington’s quality work is the promotion of a shared understanding of high-quality program practice. Providers in the ELOQI valued this shared understanding, as well as the provision of relevant data and coaching support.^[6] School’s Out Washington has also identified several barriers to quality improvement work. Smaller organizations with fewer resources have difficulty finding time for coaching, training, and data collection. Staff turnover and a lack of stable, predictable funding are persistent challenges across the field. School’s Out Washington is exploring new and innovative ways to help address these barriers.

Data is key

We know that data is key to designing and improving systems to have the most viable impact on young people. However, getting that data, maintaining it and using it effectively is a persistent challenge. Across our sector, there are multiple systems, projects and approaches that are not fully aligned—and, in many cases operating in silos. At the organization and program levels, data collection tools and management practices vary significantly. Within districts and schools, educators lack visibility into programs serving kids in their communities and in their own buildings. Conversely, many providers cannot access geographic and comparative data that would help them understand the programming landscape and improve service delivery.

With the launch of Elevate Washington, we’ve established a foundational data system that can help us overcome our knowledge deficit and move the expanded learning field toward more agile, data-informed decision making, advocacy, and direct-service impact. 

Looking ahead

As we look toward the future, we remain committed to providing Washington’s young people with exceptional places to learn and grow before and after school, and during the summer months. At the same time, we need to ensure that the people providing these learning experiences are equipped with the knowledge, skills, and professional support they need to do their best work. Finally, we remain committed to building an equitable and accessible expanded learning ecosystem. In order to do this, we are working to build a shared vision for this ecosystem than spans Washington’s localities and transcends sectors. As School’s Out Washington defines its own role in this ecosystem, we want to prioritize the full and equitable engagement of People of Color (POC)-led organizations and those embedded in high-poverty, underserved communities.

^{[1] Pierce, K.M., Auger, A. and Vandell, D.L. (April, 2013). Narrowing the Achievement Gap: Consistency and Intensity of Structured Activities During Elementary School. Unpublished paper presented at the Society for Research in Child Development Biennial Meeting, Seattle, WA.}

^{[2] Durlak, J. A., Weissberg, R. P. & Pachan, M. K. (2010). A meta-analysis of afterschool programs that seek to promote personal and social skills in children and adolescents. American Journal Community Psychology, 16, 294-309. doi: 10.1007/s10464-010-9300-6.}

^{[3] Smith, C., Roy, L., Peck, S. C., Macleod, C., Helegda, K., & Hughes, J. (2018). Quality-outcomes study for Seattle Public Schools summer programs, 2016 program cycle. Ypsilanti, MI: David P. Weikart Center for Youth Program Quality.}

^{[4] Joseph, G. E., Branson Thayer, M., Semu, B., Hassairi, N., Zeng, S., & Porter, A. (2017). Expanded Learning Opportunity Quality Initiative Pilot Evaluation Study Report. Seattle, Washington: Cultivate Learning at the University of Washington.}

^{[5] Schilder, D. (2019). Washington’s Expanded Learning Opportunities: The Promise of a Systems Approach. Boston, MA: Build Initiative.}

^{[6] Washington State Department of Early Learning and Office of the Superintendent of Public Instruction. (2017). Expanded Learning Opportunities Quality Initiative: Report to the Legislature.}