Areas of Expertise - Sample Project Descriptions

Workforce Development

Evaluation of the Implementation of the Workforce Investment Act (WIA) of 1998

WIA represents a massive overhaul of the nation's workforce development system, with an emphasis on streamlining employment and training services, fostering linkages and coordination across categorically-funded programs, and promoting customer choice. SPR was contracted by DOL to study how WIA is being implemented across the nation. This includes looking at the effectiveness of partnership formation between job training agencies, the Employment Service, school-to-work systems, and others programs. The study consists of two major components, including intensive site visits to WIA systems and partner agencies across the nation and the analysis of data gathered from a national implementation tracking system. The study also includes a component, funded by the National School To Work (STW) Office, that examines the extent to which WIA and STW efforts are linked.

This evaluation attempts to estimate the impact and cost-benefit of the use of self-directed labor exchange services, or those not requiring registration under either the Workforce Investment Act or Wagner-Peyser. Data is being drawn from administrative records, including Unemployment Insurance claimant files, Unemployment Insurance wage files, and Employment Services and WIA participant records, as well as from surveys of local workforce agencies, users of self-directed services, and employers. The project also entails a qualitative component, with site visits to selected local One-Stop Centers, so that the quality of self-directed services can be assessed.

WIA Performance Outcomes Technical Assistance and Training Performance Enhancement Project

The purpose of this project is to provide technical assistance to enhance the performance outcomes under WIA Title IB programs for selected states and local areas. SPR, in collaboration with co-contractor Public/Private Ventures and in close consultation with Department of Labor regional staff, has provided collective and individualized technical assistance to regions, states, and local areas to enhance outcomes for the core and customer satisfaction performance measures for adult, dislocated worker, and youth programs. SPR facilitated consultations with regional staff, developed training and technical assistance workshops and materials, provided collective and individualized technical assistance to states and local areas, and will complete a final report on technical assistance activities.

WIA establishes a performance accountability system to assess the effectiveness of states and local areas in achieving continuous improvement of workforce investment activities. SPR was awarded a contract to evaluate the effectiveness of the current system as it relates to continuous improvement in workforce investment activities and identify alternative measures that might more effectively accomplish the aims of the system. To accomplish these goals, SPR is taking a multi-pronged approach that includes: the compilation of data from a sequence of workgroups that have met on workforce performance accountability over the past half-decade, site visits to 15 states and 25 local areas, a survey administered to all Local Workforce Investment Areas (LWIAs), and a quantitative analysis of data on WIA program participants.

Evaluation of Worker Profiling and Reemployment Services (WPRS) Systems

This evaluation was conducted to provide the Department of Labor (DOL) and Congress with information on the development, implementation, and effectiveness of WPRS systems. The goals of the evaluation were to determine: (1) how well profiling procedures work in identifying claimants at risk of exhausting their Unemployment Insurance (UI) benefits and (2) how effective early intervention with reemployment services works in helping profiled UI claimants return to productive employment. The evaluation consisted of two components: an implementation and process study, and an effectiveness study. The implementation and process analysis was based on site visits to six prototype states and a nationwide survey of administrators involved in the WPRS initiative. The effectiveness evaluation included two components: (1) an analysis of aggregate state-level data derived from the administrative surveys and data submitted by states to DOL, and (2) an analysis of claimant-level data from selected states using a comparison-group methodology. A customer satisfaction survey was also conducted to provide DOL with baseline information on claimants' satisfaction with reemployment services.

National Job Corps Capacity Building Project

The objective of this project, conducted for the U.S. Department of Labor/Office of Job Corps, was to support the Job Corps system in its efforts to coordinate with state and local workforce development systems and employers, and to enhance community involvement. SPR identified and disseminated successful models and best practices for Job Corps centers to link with One-Stop career centers, School-to-Work initiatives, state and local education and workforce development systems, welfare reform initiatives, employers, and community representatives. Project activities included a review of existing workforce development and educational initiatives, telephone interviews with 40 Job Corps centers to identify best practices, and site visits to 10 Job Corps centers. Thereafter, SPR staff provided technical assistance to Job Corps centers to develop models of linkages and collaboration, and prepared a report containing policy recommendations and best practices for increasing Job Corps' involvement with workforce development systems, educational systems, employers, and the local community.

Evaluation of the Individual Training Account (ITA) Demonstration

ITAs represent an important new strategy under the Workforce Investment Act for promoting customer choice with respect to training services. This study, conducted for the U.S. Department of Labor, is designed to shed light on how ITAs are administered and to assess the relative effectiveness of alternative strategies for their use. The study has two major components. The first component consists of a process study of 13 pilot projects throughout the country that designed and implemented ITAs; the second will consist of a process study of additional sites as well as an experimental evaluation of the different ITA "treatment" designs that they will be implementing. SPR is undertaking this project with two other firms and has primary responsibility for the process study associated with both of the study's components.

Evaluation of School-to-Work Out-of-School Youth Demonstration and Job Corps Model Centers

This implementation and process study examined what adaptations of the school-to-work model needed to be made to accommodate the special circumstances involved in serving out-of-school youth. Projects we studied included 11 specially-funded demonstration programs and 30 Job Corps Model Centers. Data were collected during two waves of intensive site visits and interviews with program administrators, classroom instructors, planners, employers, participants, and partner agencies.

Tri-County Community Asset Investment Portfolio

As part of an overall effort to enhance their strategic planning, the local Workforce Investment Boards (WIBs) of the counties of Santa Cruz, Monterey, and San Benito in the Central Coast area of California funded a Community Assets Investment Portfolio to assist local stakeholders identify the strengths and weaknesses of their workforce development system and design a system that is responsive to the needs of current and potential customers. SPR worked with the three local WIBs to help them develop an accurate understanding of the characteristics of job seekers, incumbent workers, and employers, as well as the resources that are available to address their workforce development needs. This assistance was primarily focused on the development of four reports: a Community Needs Assessment, a Community Inventory of Services using a web survey, a Gap Analysis, and a final Summary Portfolio which summarized the overall results of the project.

Youth Opportunity grants were awarded to 36 urban and rural communities across the nation to attack the persistent problems of poor labor market success and associated social ills plaguing youth in high-poverty areas. The grantees are charged with mobilizing resources in high-poverty neighborhoods in a sustained, five-year effort to provide a diverse array of services, including after-school programs, tutoring, recreation programs, job training, and work readiness training. The five-year evaluation for the U.S. Department of Labor studies the implementation of YO, measures the impact of the program on youths' employment, educational attainment, welfare dependency, health status, and involvement in crime, and assesses the target areas' sense of well-being before and after receiving program services. To achieve these objectives, the evaluation involves a process study, MIS data analysis, a survey of local youth, and an ethnographic study.

Social Policy Research Associates (SPR) and its subcontractor are examining the operation and impact of the Trade Adjustment Assistance program. The five-year evaluation consists of two major components: (1) a process study entailing multi-day site visits to selected states and local areas, as well as a survey of all local areas throughout the country, and (2) a quantitative analysis of administrative data and survey data on TAA participants and members of a comparison group. By conducting detailed case studies of a selected number of states and local areas over the five years of the evaluation, the research team will gain an in-depth understanding of how the TAA program operates in practice. A particular focus of the first round of site visits will be on how states and local areas have responded to the mandates of the TAA Reform Act of 2002 and the implementation challenges they have encountered in doing so. Subsequent rounds of site visits will focus on on-going program operations and will highlight innovative and promising practices. These site visits will be complemented by a local-area survey, to provide a snapshot of TAA program administration and operations nationwide. The evaluation team will also collect and analyze administrative and survey data on TAA participants and a comparison group of nonparticipants. The objectives of these analyses will be to characterize the program with respect to the take-up rates for services, examine reasons for participation and nonparticipation, and estimate the impact of the TAA program on training completion and post-program employment and earnings.

Evaluating Unemployment Insurance In A One-Stop Environment

This evaluation explored the connections between unemployment insurance and the One-Stop system in a range of settings to identify best practices as well as issues that need more attention. The study gathered information on methods by which UI and/or One-Stop staff handle (1) various elements of the claims process; (2) Worker Profiling And Reemployment Services (WPRS); (3) employer services; (4) design of the UI-One-Stop interface; (5) sharing customer and management information; and (6) staff training. The study placed particular emphasis on the impact of telephone claims-taking and WPRS. Evaluation activities included telephone data collection on systems and practices in UI and One-Stop administrators in eight states and eight local areas in those states and three local site visits. The site visits included focus groups of employees, claimants, and employers. The evaluation resulted in recommendations for improving UI services to claimants and employers within the One-stop system and improving the connectivity between UI and other One-Stop services.

For this project, Social Policy Research Associates (SPR) conducted an analysis of the formula that states use to allocate funds to local workforce investment areas under the Workforce Investment Act (WIA) Title I adult, youth, and dislocated worker programs. Under WIA, funds for both the adult and youth programs are to be allocated primarily using formulae specified in the Act itself. These formulae are very similar to those used in the Job Training and Partnership Act (JTPA), which WIA replaces. But under WIA, states are allowed to allocate up to thirty percent of their adult and youth funding using alternative criteria than those specified in the Act. Thus, although many states continue to allocate funds exactly as specified under WIA, other states have adopted additional allocation criteria, obtaining results that differ, perhaps markedly, from those they would have received under JTPA. This may have ramifications for the organizational and financial stability of local workforce investment boards (WIBs), or for states' ability to sufficiently achieve the goals and objectives of the Act. Indeed, such alternative mechanisms may better enable states to focus their WIA funds to those local areas most in need or those that are most effective in serving clients.

To address these questions, and others, including the extent to which states are using any alternative criteria in allocating funds, we used a multiphase research agenda that included qualitative and quantitative methodologies and enabled us to compare not only the allocations that were being made to local areas, but also many of the potential alternatives identified previously to assess how these alternative formulae impact the allocation to local areas and whether some of these alternatives produce a result viewed as more favorable.

The objective of this project is to collect, analyze, and disseminate information on how state and local One-Stop systems have involved partners from programs and agencies beyond the five mandated programs funded by the Department of Labor (DOL). The project focuses particularly on services to youth from the DOL-funded programs working in partnership with education agencies, service corps, and community-based organizations.

Using site visits to five local One-Stop systems that have had experience involving a broad range of partners in One-Stop planning and career center operations, we will identify approaches that appear to be successful in building broad partnerships and analyze both extent that broad partnerships have strengthened One-Stop systems (e.g., through increased scope and visibility, increased total resources, and the capacity to provide more diversified services to job seekers and employers). We will also document the special challenges associated with broad One-Stop partnerships, such as a greater difficulty in achieving consensus on center goals and priorities and increased complexity of cross staffing and service consolidation efforts.

As a written product for this project, we will prepare a guide on How to Develop and Maintain Effective One-Stop Partnerships. As part of the project, we will also develop briefing materials and hold workshops at One-Stop conferences on building effective partnerships with a wide variety of agencies and programs.

The project also included the pilot-testing of a modular training program, Partnering for Quality, designed to help local workforce investment partners meet the requirements of the Workforce Investment Act (WIA) for an integrated One-Stop service system. The training provided participants with tools, strategies, and case study examples on (1) developing broad partnerships, (2) designing seamless services that meet customer needs, and (3) managing integrated One-Stop systems. The training modules are designed to address specific requirements of the Workforce Investment Act (WIA).

Youth Development

Evaluation of the Ford Foundation's Youth Leadership Development Initiative (YLDI)

The goals of this two-year evaluation were to (1) assess the degree to which civic activism is an effective youth development strategy for reaching vulnerable youth populations, (2) identify organizational practices that promote positive youth development outcomes, (3) strengthen the field of youth development through the lessons and best practices of YLDI grantees, and (4) disseminate lessons learned to the larger youth development and funding communities, as well as policymakers. Project activities included conducting two rounds of site visits, administration of a youth survey, providing ongoing technical assistance, culling the learning taking place across the initiative's core partners, reviewing relevant youth development and civic activism literature, and publishing and disseminating results to the youth development field.

This project was a partnership among eight innovative youth-serving agencies, six funders, SPR, and the Community Network for Youth Development (CNYD). Together these partners developed a clearer understanding of youth development outcomes, how to measure them, how to build the capacity of the youth-serving agencies to conduct self-assessments, and how funders can improve their grantmaking. SPR partnered with the CNYD to conduct this pilot project, which included forming a learning group, developing assessment tools and resources, and strengthening agencies' capacity to improve their own accountability systems around quality youth development practices and program outcomes.

This two-year study of six high schools in California examined the effects of contextual influences and the impact of secondary school interventions on intergroup relations. Specifically, the study addressed two research topics: (1) influence of family values, peer norms, and school context on students' relations with youth of different backgrounds; and (2) effects of systematic school strategies on improving intergroup relations. The interventions that SPR studied aimed to improve youth's ability to cross ethnic and racial borders, interact with members of different groups, and identify commonalties.

Civic Activism Among Marginalized Youth

This project, funded by the Pew Foundation's Center for Information and Research on Civic Learning and Engagement (CIRCLE), examined youth organizing and identity support as key strategies for engaging marginalized young people in civic life. In the first year of the study, we conducted a rigorous and in-depth survey analysis, comparing 12 civic activism organizations with 12 "traditional" youth development organizations. In the second year of our study we conducted site visits to 12 additional organizations that focus explicitly on issues of identity within the context of youth organizing. This qualitative piece allowed us to provide a richer and more in-depth context for understanding our survey results. Additionally, we worked with other organizations to develop alumni networks and a program alumni survey in order to track civic participation for program "graduates" over time.

This study, conducted on behalf of the Surdna Foundation and the Open Society Institute, was developed to better understand the impact of youth media on the audiences that it reaches. Guiding questions include: What is an appropriate framework and methodological approach for measuring youth media's impact across youth producers, youth and adult audiences, adult media, and channels of distribution? What might be a threshold level of critical mass impact? Conceptualized in two phases, the first phase focuses on developing a field-generated multi-level youth media outcomes framework, as well as identifying an appropriate evaluation method and design. Phase II will then systematically measure the collective impact of youth media on audiences within one region of the United States.

SPR conducted a three-year evaluation of the Vulnerable Youth (VY) grantmaking strategy within the C.S. Mott Foundation's Success in School funding area. This study—focused on grantmaking process and lessons learned—operated at the intersection of three primary fields: youth development, education, and workforce development. For the first two years, we focused on VY grantees' lessons learned and promising practices with regard to serving vulnerable youth populations. Lessons centered on such topics as: identifying key elements of programmatic success in alternative education schools; bridging the gap between alternative and mainstream education systems; addressing issues of racism and diversity; and identifying organizing principles for the VY field. For the third and last year, we focused on the extent to which VY grantees were demonstrating progress toward identified Foundation benchmarks. The broad aim of the overall study was to help inform the Foundation's future VY grantmaking and its efforts to build the field. Our data collection efforts consisted mainly of grantee interviews, site visits, and document review.

This evaluation for the Cowell Foundation examined their youth grantmaking. The primary objectives of this project were to assess: (1) the extent to which grantees' program goals matched with those of the foundation and (2) the capacity of grantees to report information that the foundation needs to measure youth competency results. Core activities of this project included conducting site visits and interviewing 10 grantees on their goals, data collection capacity, and quality of existing data on youth outcomes. The study also included a review of existing literature on measuring youth outcomes. These activities culminated in a set of recommendations to the foundation about what can be reasonably expected of grantees in terms of their project goals and their documentation of success in meeting youth's developmental needs.

The Workforce Investment Act (WIA) requires local workforce areas to dedicate at least 30% of their overall youth funds to Out-of-School Youth (OSY). However, local areas report considerable difficulty in recruiting, enrolling and retaining OSY for a variety of reasons, including the targeted youth's weak attachment to traditional institutions. This project was funded by the U.S. Department of Labor to identify and learn from local areas that demonstrate "best practices" in enrolling and serving OSY, and from others that are experiencing particular challenges. The study consisted of multi-day site visits to eight local areas to learn from their efforts, and culminated in a report that described their experiences.

SPR was funded by the Innovation Center (IC) for Community and Youth Development to engage a select group of civic activism organizations in work on evaluation and planning. Our work plan includes four primary components, including: (1) assistance with youth development survey interpretation and analysis, (2) targeted technical assistance on self-evaluation, (3) convening of an evaluation learning group, and (4) development of a curriculum on self-evaluations. These elements reflect the specific needs identified by participating civic activism organizations. Lessons from this effort will feed into an online resource for youth organizations called the Planning and Evaluation Resource Center (PERC). Our work will highlight the principles of civic activism, with an eye towards developing indicators, outcomes, and evaluation instruments to advance the field.

Education

The evaluation for the Pathways to College Network (Network)—a large collaborative of foundations, non-profit organizations, educational institutions, and the U.S. Department of Education around improving access to higher education among traditionally underserved youth—is both a process and an outcome study. The study examines and makes recommendations on goal refinement, administrative processes, governance, and inter-partner communication for the Network as a collaborative. In addition, the evaluation is measuring partners' progress toward their goals as well as those of the Network, and is assessing the effects of Network activities on multiple levels. For instance, SPR is examining how Network activities and products are influencing the knowledge and behaviors of partner organizations.

This two-year formative and summative evaluation explored how ATLAS reforms impacted both teaching and learning by studying implementation of the model and the impact of reform efforts on students, teachers, and schools. The study included 16 schools in four school districts nationwide that were implementing ATLAS. The study aimed to understand how ATLAS reforms were implemented, how well they functioned and to what extent they led to concrete changes in both instructional and institutional practices. The study examined the relationship between implementation, impact on practice, and outcomes at the student, teacher and school level. SPR's mixed-methods approach included qualitative site visits to each of the four school districts, a survey of teachers in all 16 ATLAS schools, and an analysis of student achievement data.

Funded by the state of Washington Workforce Training and Education Coordinating Board, this project entailed an estimation of the impact of the state's School-to-Work (STW) initiative on exiting cohorts of high school students. Impacts were estimated on an array of student outcomes, including their employment and earnings and post-secondary school attendance. All outcomes were measured approximately one year after the students left high school.

This evaluation was requested by the James Irvine Foundation to assess the goals and impact of the Forum for the Future of Higher Education—a Foundation grantee that serves as a learning community for leaders from select colleges and universities across the country. The Forum's primary activities are its sponsored research, annual symposium, and publications. SPR interviewed Forum leadership and California-based participants in order to determine how well their respective goals aligned, and to gauge the effect of the Forum's research activities and publications on the individual, institutional, and field levels. SPR also provided recommendations for improvement of the Forum's activities and publications.

The NHSA is a partnership of over forty organizations committed to promoting the academic achievement and social development of all high school-aged youth by building opportunities for shared learning and concerted action; informing practice, policy and research at the local, state and federal levels; and increasing public awareness to support the development of effective practices and policies. SPR's two-year, formative and summative evaluation of the NHSA has five key goals: (1) understand the NHSA's theory of change, (2) analyze the NHSA's collaborative structure in order to understand its effect on progress, (3) assess the extent to which the NHSA completes its planned activities and progresses toward its three key goals, (4) assess the NHSA's effect on its member organizations, and (5) distill the NHSA's primary challenges and factors of success.

For this project, SPR evaluated the Hewlett Foundation's work and impact on Bay Area K-12 education reform and provided recommendations for future directions in this area. SPR assessed the Foundation's K-12 education grantmaking process and grantmaking impact. SPR also examined the Foundation's grantmaking role in the context of Bay Area education reform. SPR gathered the insights of education leaders, grantees, and funders on the level and nature of the impact being made, and on the quality of the relationships being built among these different constituencies. This work included discussing whether the Foundation's targeted areas were the most effective ones, the extent to which grants were achieving the strategic goals, and proposing ways to hold grantees more accountable.

SPR conducted a three-year evaluation of the Community-Driven School Reform grantmaking strategy within the C.S. Mott Foundation's Success in School funding area. The grantmaking strategy focused on community organizing as a vehicle for harnessing the power of low income communities to improve their local schools and school systems. In addition to a focused study on the grantmaking processes of this funding area, our study captured lessons learned and promising practices of grantees within the CDR portfolio. Through grant review, interviews and site visits, SPR: 1) assessed the progress of the overall CDR funding strategy and made key recommendations, 2) documented lessons learned and promising practices, as well as 3) completed mini-studies on the role of youth in CDR, CDR at the high school level, and how community organizing for school reform differs from community organizing in more traditional issue areas.

This evaluation was requested by the W. K. Kellogg Foundation to analyze the extent to which their grantmaking in the higher education field met the goals articulated by the Foundation's Higher Education Strategic Plan. Specifically, SPR assessed the grants made under the Foundation's four primary higher education strategies to examine the progress grantees made with regard to their proposed activities, to assess how effectively grantees met the expectations of various stakeholders, and to distill critical lessons and best practices to be applied to future higher education grantmaking efforts. Two of the strategic areas we evaluated were specifically concerned with educating a more diverse student body and promoting partnerships between mainstream and historically minority institutions. The project produced multiple reports on (1) the extent to which the Foundation's funding has had an impact at the individual, institutional, community, and policy levels, (2) the key lessons learned from project implementation, and (3) the integration of the higher education grantmaking with the Foundation's identified Cross-Cutting Themes.

School Equity Study

This study, funded by the Rockefeller Foundation, was designed to inform educators and policymakers about the conditions of California's public schools, define the basic resources that all schools need to meet state requirements, and identify any areas where there was a discrepancy between what schools have and what they are expected to achieve. SPR conducted site visits to 19 schools in rural and urban communities throughout California. The site visits were designed to inform our study about how the physical, environmental and material conditions of public schools influence their ability to meet their educational mission. The site visits were designed to complement a teacher survey. The study culminated in a report that included school profiles as well as a cross-case analysis designed to inform policymakers and others about how the overall conditions of public schools impact their capacity to provide high-quality teaching and learning environments.

The MarcoPolo Multicultural and Multilingual Needs Assessment Project

Dedicated to providing high quality Internet content and professional development to teachers and students throughout the United States, MarcoPolo: Internet Content for the Classroom is a nonprofit consortium made up of premier national and international education organizations and the MCI/Verizon Foundation. In the MarcoPolo Multicultural and Multilingual Needs Assessment Project (MCML), SPR critically analyzed the educational content and professional development materials and courses available on the MarcoPolo curriculum website from the perspective of multicultural and multilingual education populations. Through a comprehensive needs assessment, we assessed how well the existing educational content and professional development adequately represent, reach and provide for multicultural populations. As a result of this assessment, we provided detailed recommendations for adapting, augmenting and/or supplementing MarcoPolo’s educational content and professional development toward the goal of providing the highest quality Internet-based education resources and professional development for multicultural and multilingual education populations.

Family Support

The State of California requested this evaluation to determine how well ten pilot counties' CELs—consolidated lists of families requesting subsidized child care—met the needs of state, county and local governments for child care funding and planning purposes. The evaluation examined which CEL methods work best in which settings to gauge the true demand for subsidized child care. The evaluation utilized both qualitative and quantitative methods to inform the State of California, Child Development Division, Department of Finance, the Legislature, and the child care community on the following issues: processes and impact of CEL system implementation; impact of CEL systems on families eligible for subsidized child care; impact of CEL systems on subsidized child care agencies; impact of CEL systems on the community and state levels; CEL costs, benefits, and best practices; and the potential of CEL models to facilitate child care planning and policymaking at the state, county, and local levels.

Evaluating the Quality Child Care Initiative: Child Care Worker Apprenticeships

This evaluation is a process study of the U.S. Department of Labor's effort to apply apprenticeship training methods to the child care field in order to increase wages and skills of child care workers. The initiative is designed to address the major problem in the field: low quality of child care. The process study focused on 10 western states. The project also prepared a statistical picture of the program participants and developed estimates of sustainability after the grant ends in 21 states throughout the country. Technical assistance was provided through a subcontract with the Center for Career Development in Early Care and Education at Wheelock College.

The objective of this project was to identify and disseminate effective program models for collaboration between One-Stop workforce development systems and welfare-to-work systems in meeting the challenges of welfare reform. Project activities included a review of existing information and contacts with the sixteen first- and second-round One-Stop Implementation Grant states, and the completion of case studies of six sites with well-developed One-Stop and welfare-to-work linkages. An initial cross-site analysis led to a Review of State Welfare Designs and Delivery Systems. After analyzing the case studies, project staff prepared a guide on Local Models for One-Stop Collaboration with Welfare Reform.

Evaluation of the Bay Area Early Childhood Funders' Quality Child Care Initiative

This three-year evaluation, requested by the Leadership Team of the Quality Child Care Initiative (QCCI), tracked the activities and accomplishments of the Initiative, which was a collaborative effort by over 22 foundations in the ten-county San Francisco Bay Area to improve access to quality child care by low-income working families. The evaluation addressed key questions about the design, implementation, and effectiveness of the varied activities undertaken by QCCI funders, strategic partnerships, and project grantees to further four strategic objectives: (1) improving and creating new child care facilities; (2) supporting advocacy for quality child care, (3) supporting the creation of a stable child care workforce through training and increased compensation initiatives, and (4) increasing access to child care by supporting the creation of centralized eligibility lists for subsidized child care. Data for the evaluation was collected using a variety of methods, including document review, telephone interviews, and focus groups.

SPR is designing the Parent Services Project Toolkit to offer guidance and concrete support to early childhood practitioners and parents who want to answer two basic questions: (1) How are we doing? and, (2) Are we making a difference? The Toolkit is meant to be user-friendly and useful in addressing evaluation questions that are frequently asked by family support programs. The product incorporates the guiding principles of family support in the evaluation process. Because all family support programs are not the same, this Toolkit best represents a strategy for getting started on evaluation work. The Toolkit allows practitioners to select specific items, components and examples that best match their program, and modify them to meet particular evaluation needs. The connection between evaluation and the principles of the family support, particularly the idea of partnership and shared power, informs much of the content in the Toolkit.

SPR was awarded a grant by Food and Consumer Service (FCS) to examine how, and how many, Food Stamp recipients are being served in JTPA. Using JTPA's Standardized Program Information Report (SPIR), SPR examined how many Food Stamp recipients were being served in JTPA, what services they received, and what outcomes they achieved at exit. Since Food Stamp work registrants are the particular focus of the E&T program's efforts, we examined separately the experiences of Food Stamp recipients who also received AFDC from those who received Food Stamps but not AFDC. For purposes of comparison, the Survey of Income and Program Participation (SIPP) was used to track the employment experiences of Food Stamp work registrants more generally, almost all of whom presumably were not enrolled in JTPA or receiving other training services. Results suggested that substantial numbers of JTPA participants were Food Stamp recipients, that they received intensive employment and training services on average, and that many attained unsubsidized employment upon exiting from the program. We suggested that the Food Stamp E&T program and JTPA could be natural partners for coordination, with JTPA providing access to intensive training services for Food Stamp work registrants who are deemed hard to serve, and with the Food Stamp E&T program providing JTPA with a steady client pool.

Diversity & Equity

Evaluation of the Kellogg Foundation's Capitalizing on Diversity Cross-Cutting Theme

The primary goal of this multi-year project is to support the W. K. Kellogg Foundation's Capitalizing on Diversity Cross-Cutting Theme and the foundation's efforts to integrate diversity throughout its grantmaking. In the first year, SPR provided the foundation with a preliminary assessment of the impact of the Capitalizing on Diversity Cross-Cutting Theme and developed a learning plan for integration of diversity at the foundation by 2004. Since then, we have provided ongoing technical assistance towards achieving diversity integration goals through such project activities as: developing conceptual frameworks of diversity grantmaking, supporting the development of integration benchmarks and indicators, facilitating foundation-wide dialogue, producing case studies on diversity grantmaking, creating a diversity grantmaking toolkit, serving on advisory committees for foundation initiatives, conducing research on immigrant populations, and leading trainings on integration of diversity in grantmaking.

The California Endowment enlisted SPR to support foundation efforts in building the capacity of the health evaluation field to utilize multicultural approaches needed for building and sustaining healthy communities throughout California. Over the course of this two year project, we worked to: (1) expand the Endowment's network of diverse evaluators to better reflect and meet the needs of program staff, grantees, and the evaluation department; and (2) enhance available resources through publications and convenings on the incorporation of diversity into health evaluations.

This project entailed four main tasks:

1. Comprehensive needs assessment of The California Endowment's evaluation requirements, through which we articulated effective strategies for strengthening The Endowment's evaluation network. This included interviews with key Endowment staff, grantees and partners as well as review of key documents.
   
   
2. Identify Evaluation Consultants. We expanded The Endowment's network of evaluators through strategically recruiting prospective evaluation consultants who can meet the needs of The Endowment's grantees.
   
   
3. Analysis of Diversity in the Health Evaluation field, by conducting an analysis of the health evaluation field with a special focus on identifying qualified diversity evaluators and diversity evaluation approaches, we (1) developed a resource guide and summary of key findings and recommendations that can immediately support health practitioners and evaluators in addressing diversity within their work, and (2) identified leadership opportunities for The California Endowment in developing the capacity of the health evaluation field to address diversity issues.
   
   
4. Finally, drawing upon information and relationships gained throughout the project, we support The Endowment as it takes a leadership role in convening regional roundtables aimed at developing the capacity of the evaluation field and its partners to work at the intersection between health and diversity.

The goal of this evaluation is to support WFN's efforts to unleash philanthropy for women and girls' funds. We will evaluate WFN's technical assistance efforts to increase its members' capacity to increase their assets and provide ongoing feedback and technical assistance on WFN's self-evaluation goals. Additionally, we will assess the extent to which creating a comprehensive Knowledge Management System is an effective and sustainable approach to sharing information among members to enhance their fundraising capacity. We will use a formative and summative approach to answer the core research questions on (1) Increasing the Capacity of Member Funds, (2) Strengthening Network for Learning, and (3) Increasing the Funding Base. The research questions include, how accessible, useful, and applicable are the knowledge, tool and TA to member practices? How does WFN function as a network? How does WFN build off the existing strengths of WFN and member funds? Do the capacity of member funds change as a result of WFN interventions? Which approaches or combinations of strategies are most effective in increasing members' capacity? What are the lessons learned from WFN's capacity building efforts? How does knowledge management enable WFN to function better as a learning network? How effective is WFN in convening, collaborating and leveraging knowledge to learn from each other? What changes in funds/philanthropic assets and giving occur as a result of WFN's efforts? How are WFN's capacity building efforts affected the role of women of color within WFN's member organizations?

Project activities include documentation and assessment of WFN's capacity building efforts, technical assistance, site visits to member funds, and an annual survey to member funds.

This two-year study of six high schools in California examined the effects of contextual influences and the impact of secondary school interventions on intergroup relations. Specifically, the study addressed two research topics: (1) influence of family values, peer norms, and school context on students' relations with youth of different backgrounds; and (2) effects of systematic school strategies on improving intergroup relations. The interventions that SPR studied aimed to improve youth's ability to cross ethnic and racial borders, interact with members of different groups, and identify commonalties.

The goal of this national evaluation is to guide the Kellogg Foundation and grantee efforts to support positive youth outcomes around working against racism in their communities. In addition, the project will distill lessons with regard to how this initiative can serve as a model for similar work by the foundation to support diversity grantmaking. Our evaluation framework draws on youth development principles, prior work on intergroup relations, and organizational change theory in order to articulate a theory of change at the foundation, community, program and individual levels. Project activities include documentation of the implementation process, evaluative technical assistance, site visits to grantees, and a baseline survey of grantees.

This exploratory study, commissioned by The San Francisco Foundation and The Walter and Elise Haas Fund, examined educational experiences that lead youth to greater respect for diversity and a sense of being a part of the larger whole. Specifically, the study looked at: (1) the measures educators take to ensure that students who are starting from different places (nationally, linguistically, and experientially) find common ground with their peers; (2) the role that schools play in promoting or supporting intergroup friendships and student activism on civic issues; and finally (3) factors that prevent or facilitate students' embracing of ideas and values consistent with a participatory democracy.

This project evaluated the program practices of a locally driven environmental leadership program for culturally diverse high school youth. The purpose of the evaluation was to examine outcomes for youth as they transitioned from secondary school to postsecondary education and employment opportunities. The study was designed to measure the effectiveness of the program goals of: (1) broadening the environmental movement by embracing cultural diversity, and (2) providing a nationally replicable program enabling youth from diverse backgrounds to acquire leadership skills related to issues of environmental protection and social justice. Study methods included a qualitative process evaluation of the program over a one-year period based on a conceptual model of effective "youth based services," and a quantitative impact study of program outcomes.

The Evaluation of the Title IV Indian and Native American Program

This two-year study for the U.S. Department of Labor evaluated the strengths and weaknesses of the Indian and Native American Program and its ability to be responsive to the needs of diverse Indian peoples and communities. In collecting data for the study, staff members conducted intensive case-study visits to 23 randomly-selected grantees, during which interviews were conducted with program administrators and staff, service providers, influential community members, and current and past program participants. Additionally, a survey was administered to all grantees, eliciting information about program priorities, service designs and service strategies, and grantees' views about how the Indian and Native American Program could be made more effective.

SPR was contracted by the NALEO Educational Fund to conduct an evaluation of nine California Policy Institutes on Health. The NALEO Educational Fund has received a grant from the California Endowment and will conduct these Policy Institutes for Latino elected and appointed officials around the state. These institutes are one-day training and leadership building events that will explore public health policy priorities and enhance the skills of participants to address and improve Latino access to quality health systems and resources. SPR is conducting a three-pronged evaluation to assess the effectiveness of these Policy Institutes and to provide NALEO with: (1) information about learning gains by participants of the Policy Institutes, (2) customer feedback on the effectiveness of its Policy Institutes, and (3) feedback about the degree to which participation in the institutes affected the behaviors, attitudes, and effectiveness of participating officials.

The goal of this evaluation was to capture community members' perspective of the benefits and impact of the Wildflowers Institute's asset-based approach to engaging different ethnic communities. The Wildflowers Institute designed an innovative framework to uncover cultural community patterns, as well as developed processes to assist individuals in explicitly articulating the unspoken premises of each community's culturally-based leadership formations and social patterns. Our research included site visits, focus groups, and interviews with elders, leaders, and youth in five Asian Pacific American groups in Northern California, including the Lao Iu Mien, Lao/Khmu, Cambodian, Chinese, and Filipino communities. Participant feedback included articulating ways in which Wildflowers' approach has influenced philanthropy's approach to grantmaking in different ethnic and racial communities.

Evaluation of the Katrina Women's Response Fund

In response to the destruction and massive displacement caused by Hurricane Katrina, the Ms. Foundation for Women (MFW) and the Women’s Funding Network (WFN) created The Katrina Women’s Response Fund. The Katrina Women’s Response Fund focused on ensuring that the priorities of low-income women, women of color and their families were central to the Hurricane Katrina relief, recovery and rebuilding process. SPR conducted an external evaluation of the Fund and focused on outcomes arising from the grants and the approach furthered by MFW and other women’s funds, with a particular concentration on how the initiative promoted the leadership of women of color. This report highlights a number of key findings related to the leadership of women of color in community coalitions, social change grantmaking, grantmaking in response to a natural disaster, and engaging local policy makers and decision makers.

In response to the human suffering, destruction, and massive displacement caused by Hurricanes Katrina and Rita, the Twenty-First Foundation (21CF) moved rapidly to establish the Hurricane Katrina Recovery Fund (HKRF). With the support of an impressive coalition of diverse donors and partners, the Fund’s mission was to make strategic grants for relief, recovery, and rebuilding efforts that provide a voice—particularly for Blacks and low-income communities of color—in the equitable rebuilding of the region. SPR conducted an evaluation of the Fund, assessing the impact of 21CF's grantmaking and its grantees’ progress and impact in 21CF’s priority areas. We used primarily a qualitative approach to capture (1) grantee level outcomes, and (2) the unique role of the 21CF in brokering relationships and information to ensure that the priorities of Black, low-income, and other affected communities are central to the Hurricane Katrina relief, recovery, and rebuilding process.