A nationally acknowledged leader in conducting evaluations of and providing technical assistance for federally funded projects, SPR has spent over two decades studying federal, state, and local initiatives aimed at improving the efficiency of the labor market and ensuring that workers who are struggling in their careers gain the skills they need to be productive. Our evaluations have included qualitative implementation and process studies, as well as quantitative net impact and cost-benefit studies. Our work is designed to assess the efficacy of publicly-funded job training initiatives geared towards vulnerable populations, including those struggling to escape poverty, youth and other new labor market entrants, and dislocated workers. Building on our research experience, we have also provided highly-valued technical assistance to workforce practitioners to increase effectiveness of and efficiencies in the public workforce development system.
Workforce Investment System and Programs
Since the Workforce Investment Act’s (WIA) inception in 1998, SPR has been contracted by the U.S. Department of Labor multiple times to provide implementation, process, and impact studies to evaluate the efficacy of its programming.
See our work
Workforce Investment Act and WIOA Programs
- Evaluation of the Implementation of WIA. Following the enactment of WIA, the U.S. Department of Labor contracted with SPR to evaluate WIA’s implementation, operation, and progress toward meeting legislative requirements in its first several years. To do so, SPR conducted multiple rounds of site visits to more than 30 local workforce areas in two-dozen states. These multi-day visits included interviews with administrators of WIA and partner programs and One-Stop operators and staff. Site visits assessed the engagement of businesses in One-Stop planning and services, and the adequacy of One-Stop services provided to special populations, including the homeless, limited-English speakers, and migrant farmworkers.
- Evaluation of One-Stop Self Services. As mandated by WIA, informational and self-help services represent important elements of the nation’s workforce development system and are key to providing universal access to workforce services. In an effort to promote this vision, substantial resources were expended to develop infrastructure to support self-services and create various tools and resources to meet diverse needs. To evaluate the efficacy of the resources expended, the U.S. Department of Labor contracted with SPR to conduct an impact and cost-benefit analysis, along with a process study.
- Gold Standard Evaluation of the Workforce Investment Act. The U.S. Department of Labor contracted with SPR to evaluate the operation and effectiveness of the WIA Adult and Dislocated Worker Programs. This project uses a random assignment design to estimate the impact of WIA Adult and Dislocated Worker Program services on participants’ employment, earnings, and other outcomes.
- Evaluation of the Individual Training Account Demonstration. Individual Training Accounts (ITAs) represent an important and new way in which WIA participants are empowered to make training choices that they feel best further their career interests. This evaluation consisted of two major components: a study of 13 demonstration grantees that were awarded funds by DOL to design and implement innovative approaches to ITA systems, and an experimental evaluation of six local workforce areas that were testing three different models for ITAs that vary with respect to the extent of counseling and guidance provided to customers and the maximum allowable ITA amount. As a subcontractor to Mathematica Policy Research, SPR designed and conducted the process study associated with each of these two components.
- Office of Regional Management Training and Technical Assistance. On behalf of the U.S. Department of Labor, Office of Regional Management, SPR managed technical assistance to all six regional offices and their associated 50 states and U.S. territories. This support included 17 individual project aimed to help improve services and system supports for all the workforce service providers in the country. Deliverables included 20 in-person trainings, train-the-trainer curriculum on workforce board improvement, workshop curriculum on 28 topics related to workforce services and job training, assessment instruments for four projects related to workforce system improvement, 12 webinars, four toolkits/guides, four reports, and coaching and consultation on several projects. Deliverables for these projects are included in the reports and research section.
Career Pathways and Sector Strategies
SPR is currently evaluating four TAACCCT grants. Our experience as evaluators for these TAACCCT grant programs has given us insight into the operational challenges faced by community colleges implementing TAACCCT grants, including the role of partnerships in developing training programs that are responsive to the needs of employers; the importance of piloting new courses and programs; and the need for career pathways that will meet the needs of individuals with different levels of formal education and relevant work experience. See our work
- Career Pathways Technical Assistance Initiative and Tools On behalf of the U.S. Department of Labor, SPR managed the Career Pathways Technical Assistance Initiative, for which we facilitated the design and development of state and local career pathway systems with eleven states and two tribal communities. Emerging from this work was the development of the Six Key Elements of Career Pathways framework and the Career Pathways Toolkit.
- Career Pathways for Indian and Native American Populations – Technical Assistance & Tools For several years, SPR has supported INAP grantees in their efforts to achieve high program and participant performance. In the last few years, this support has involved developing tools and resources to support them with development of career pathways for Indian and Tribal people. See our online community of practice, the Career Pathways Framework for Indian and Native Communities, and videos of native people talking about supporting credentials and career pathways for Indian and Native Communities.
- Bellevue College Consortium’s Health e-Workforce Program TAACCCT Evaluation. SPR is working with a consortium of 9 community colleges to evaluate activities that will help teach individuals the health information technology skills that will enable them to work in nursing and allied health occupations.
- ShaleNET TAACCCT Evaluation. SPR is working with a consortium of three community colleges and their workforce development partners in Pennsylvania, Ohio, and Texas that are developing, expanding, and refining training programs to prepare individuals for jobs in oil extraction occupations by offering both short-term and longer-term training programs.
- Michigan Coalition for Advanced Manufacturing (M-CAM) TAACCCT Evaluation. Working with a consortium of eight community colleges across Michigan that emphasize basic skills attainment and job-specific training in four core classifications in manufacturing, SPR’s evaluation includes a comprehensive implementation study, an outcomes study, and a rigorous quasi-experimental impact evaluation.
- Mt. Wachusett TAACCCT Evaluation. AMMQC has selected SPR to conduct the third-party evaluation of the TAACCCT Round 3 grant. The consortium aims to transform educational delivery methods and accelerate credential attainment in the advanced manufacturing fields of mechatronics and quality. The consortium is led by Mount Wachusett Community College (Massachusetts), and comprises three other colleges: Bossier Parish CC (Louisiana), Southwest CC (Tennessee), and North Central State College (Ohio). SPR’s evaluation will include an implementation and outcome study. Due to the challenges related to using a random assignment-based experimental design, SPR has opted for a quasi-experimental design using matched comparison groups in order to estimate program impacts on program completion, post-program employment, and post-program earnings.
Improving Customer Access and Providing Strengths-Based Customer Service
- Opening Doors for Everyone: Providing Strengths-Based Customer Service at One Stop Career Centers This technical assistance and training project sought to maximize the U.S Department of Labor Employment and Training Administration’s (ETA) efforts to empower One-Stop Career Center frontline staff members in meeting the needs of underserved populations such as older workers, Indians and Native Americans, migrant and seasonal farmworkers, and persons with disabilities. Frontline staff learned strengths-based strategies to serving customers through various resources created by SPR, including webinars, a toolkit, a video, and in-person trainings. See the community of practice website containing many resources including the toolkit.
- Effective Case Management. Sponsored by U.S. Department of Labor, ETA the objective of the technical assistance was to develop tools and resources to help system administrators, local leaders and staff support high quality case management in the workforce system. For this initiative, we developed issue briefs, a framework, and assessment tools to aid practitioners – all available on the community of practice website.
Employer and Industry Engagement
- Expanding Business Engagement (EBE) Technical Assistance Initiative Sponsored by U.S. Department of Labor, ETA the objective of the technical assistance was to support grantees in their goal of improving program performance through the delivery of enhanced business-focused services stemming from a statewide business engagement strategic plan. See the toolkit and tutorial to support states in their business engagement efforts.
Improving Data Quality & Integrated Data Management
- Workforce Data Quality Initiative (WDQI). The U.S. Department of Labor (DOL) launched the DOL Workforce Data Quality Initiative (WDQI) to support states to develop and improve longitudinal workforce databases. By promoting improvements in data quality and accessibility, the WDQI aims to assist states in developing data systems that provide full portraits of individual’s educational and workforce pathways. The WDQI represents a significant effort on the part of DOL to improve access to high quality and consistent data regarding program performance.
Performance Measurement and Reporting
- Evaluation of WIA Performance Measurement System. The Workforce Investment Act (WIA) establishes a performance accountability system “to assess the effectiveness of States and local areas in achieving continuous improvement of workforce investment activities funded under this subtitle, in order to optimize the return on investment of Federal funds in statewide and local workforce investment activities.” Our evaluation of this system has two broad goals. The first is to assess the effectiveness of the current system as it relates to continuous improvement in workforce investment activities. This requires assessing how the accountability system is meeting all the explicit and implicit goals described above. The second is to identify alternative measures that might more effectively accomplish the aims of the system. To accomplish these goals, we are: (a) specifying the details of states’ and local areas’ performance accountability systems, (b) exploring their consequences, both intended and unintended, on partnerships and provider arrangements, (c) exploring their consequences, both intended and unintended, on service design and delivery, and (d) exploring their consequences for program outcomes.
- National Emergency Grant (NEG) Data Review Project. The Employment and Training Administration (ETA) has contracted with SPR to conduct a review of NEG e-system data and reporting and the integration of performance data into the operations of the NEG program. SPR staff facilitated discussions with state and local NEG administrators at each of the three 2007 ETA Performance Summits, conducted an analysis and on NEG Quarterly Progress Report (QPR) data and NEG data reported on the WIASRD, conducted telephone focus groups on NEG data issues with each of the six DOL Regional Offices and with approximately 20 grantees and authoring memos based on those findings. In addition, we assisted ETA with facilitation of a workgroup on regional and national ETA staff on developing new guidance for NEG reporting, develop webinars and training modules on NEG reporting and update our analysis of NEG data.
- American Samoa National Emergency Grants Technical Assistance Sponsored by U.S. Department of Labor, this technical assistance effort to assist ETA’s Division of Worker Dislocation and Special Response in improving the accuracy and understanding of data available under the national emergency grant (NEG) program and improving NEG performance through guidance to grantees. See the project website and Videos.
- Workforce Investment Act Standardized Record Data (WIASRD) Online Training SPR created a multi-lesson online training on the WIASRD. The training is user-friendly and introduces state and local staff to the WIASRD fields through real-life based scenarios. There are also “library” lessons where staff can look up specific fields of interest. View the training.
SPR has extensive experience evaluating a broad range of innovative workforce development programs. See our work
Innovative Programs in Workforce Development
- Workforce Innovation Fund (WIF) Demonstrations
- The Illinois Pathways Manufacturing Project (IPMP): Moving Regional Manufacturing Sector Partnerships to Scale. This four-year evaluation is being conducted for the State of Illinois Department of Commerce and Economic Opportunity (IDCEO) and its state partner agencies, the Illinois Community College Board and the Illinois Department of Employment Security. IDCEO received a U.S. Department of Labor Workforce Innovation Fund grant to develop regional partnerships with strong employer involvement and create “demand-driven” regional training programs to individuals for employment and career advancement in machine production, welding, machine maintenance, and logistics occupations. The evaluation uses a random assignment design to analyze the impacts of the Accelerated Training for Illinois Manufacturing (ATIM) program targeted to WIA-eligible adults and dislocated workers, and a quasi-experimental design to assess the impacts on participating employers. The study includes a process study drawing on site visits and a stakeholder survey, an impact study drawing on data from existing administrative sources, a quasi-experimental study based on two rounds of employer surveys, and a cost-effectiveness analysis drawing on cost data collected by the sub-grantees and the state grantee.
- Los Angeles Reconnections Academy (LARCA) WIF. This four-year evaluation is being conducted for the City of Los Angeles’s Community Development Department and Workforce Investment Board, which are recipients of a US Department of Labor Workforce Innovation Fund grant. The evaluation uses a random assignment design to analyze the impacts of the Los Angeles Reconnections Career Academy program, which targets out of school youth, ages 16 to 24, providing them educational, work readiness, supportive, training and employment services. The study includes a process study, an impact study, and a cost-effectiveness analysis. Findings from the interim implementation report can be accessed here.
- Evaluation of the Technology-Based Learning Demonstration. For this study, SPR conducted an outcomes and implementation evaluation of the U.S. Department of Labor Employment and Training Administration Technology-Based Learning (TBL) Grant program. Evaluation activities will included two rounds of site visits, including one visit to each of the TBL grantees.
- Evaluation of the STEM Demonstration. For the U.S. Department of Labor, Employment and Training Administration, SPR evaluated the design, implementation, and outcomes of five demonstration projects that prepared disadvantaged youth, dislocated workers, and other adults for jobs that require science, technology, engineering, or math (STEM) skills. The goal of the 36-month demonstration grants was to build the capacity of the participating regional consortiums to help customers interested in entering STEM fields and to help employers obtain qualified STEM workers. The evaluation identified approaches worthy of dissemination using case studies based on structured interviews with project staff, education and economic development agency partners and focus groups with participants and employers, plus an outcome study that analyzed client-level data on participant characteristics and outcomes.
- Housing and Urban Development Sustainable Communities Technical Assistance Initiative.
SPR is working with Strategic Economics, the Natural Resources Defense Council, the National Association of Development Organizations, the Institute for Sustainable Communities and others to support dozens of teams working under the US Department of Housing and Urban Development’s Sustainable Communities Grant Program. Through this work, we developed experience working across sectors and deep experience in linking workforce to broader place-based and regional strategies in urban areas and rural communities, and to connect initiatives intended to help people with special needs to mainstream development strategies. We have also begun to explore the implications of new career pathways, wealth-building, and job creation strategies linked to the sharing economy, securities crowdfunding (in states where it is legal), and entrepreneurship (and social enterprise), and new learning options including online and peer-learning platforms, new badging and credentialing systems, and the blending of work and learning through makerspaces and other shared venues. See the Project Spotlight on the Training and Technical Assistance page.
Workforce System Innovation
- Enhancing Workforce Leadership Initiative. Sponsored by the U.S. Department of Labor, Employment and Training Administration (ETA), the Enhancing Workforce Leadership project engaged nearly 500 workforce leaders at all levels in a conversation about leadership—what it is and why it matters. Using video, social media, and a series of full-immersion simulations, we helped leaders better understand their changing roles. The project culminated in the creation of the WEadership Guide and a series of related publications. See the project website at: http://enhancingworkforceleadership.org/ 2010 – 2011.
- Strengthening Workforce Boards: Tools & Technical Assistance. SPR provided technical assistance on behalf of the U.S. Department of Labor, Office of Regional Management to support workforce board chairs, members, and staff on the various roles within the workforce development system. A key goal of this project was to help board members increase their strategic approach to workforce system design within their communities. For this initiative, we developed an online toolkit, conducting trainings with board members and staff, hosted three webinars, and developed train-the-trainer curriculum – all available online.
- Workforce System Certification: Technical Assistance. SPR provided technical assistance on behalf of the U.S. Department of Labor for Region 4, supporting their workforce system with aligning services within a workforce system certification process. This strategy was designed in order to assist the various workforce programs across the states identify gaps and overlaps so that they could stretch limited funding and staff, improve customer service, and ensure that there are workers with the right skills that businesses need to grow and remain competitive. See the Toolkit, which describes Oklahoma’s Workforce System Certification.
Workforce Development for Targeted Populations
The Trade Adjustment Assistance (TAA) program provides income support, training, reemployment assistance and other services to workers who have suffered a job loss due to foreign competition.See our work
Serving Justice-Involved Youth and Adults
Please see our more information here.
Trade Adjustment Assistant Program for Displaced Workers
- Evaluation of the Trade Adjustment Assistance Program. In this evaluation, SPR examined the TAA program’s design and service delivery strategies and used quasi-experimental methods to estimate its impacts on workers’ subsequent employment and earnings. The seven-year evaluation commenced in early 2004 and drew on several data sources: (1) a process study entailing multi-day site visits to a large, nationally-representative sample of states and local areas, (2) a survey of trade-affected workers and their comparison group counterparts, and (3) administrative data (including TAA participant data and Unemployment Insurance claimant and wage data) on tens of thousands of workers in 25 states.
- Trade Adjustment Assistance Community College and Career Training Grant Program Evaluations.
Programs for Aging Workers
- Evaluation of the Senior Community Service Employment Program (SCSEP). This study for the U.S. Department of Labor, Employment and Training Administration used existing performance data from the SPARQ participant-level data system, interviews with national SCSEP grantees, and site visits to 29 local program operators to explore the organizational factors and service features that influence SCSEP program success on a wide range of outcome measures. The study focused on effective strategies to serve older workers at a time when older workers are experiencing increasing rates of unemployment and longer spells of unemployment as a result of the job losses associated with the recent recession and slow economic recovery.
- Evaluation of the Aging Worker Initiative. This multi-year project evaluated the implementation and outcomes of ten demonstration projects funded by the U.S. Department of Labor, Employment and Training Administration, to develop innovative and effective program models for serving aging workers. The funded projects were intended to recruit and train aging workers for careers in high growth industries, as well as improving the overall capacity of regional education and training systems to serve aging workers.
- NCOA Benefits Access: Evaluation of the Economic Security Initiative’s Technology Tools. SPR was contracted by the National Council on Aging (NCOA) to do a process and outcome study on its Economic Security Initiative (ESI). ESI promotes the use of comprehensive person-centered economic casework to coordinate available services to increase the economic security of older adults. This evaluation focuses on the use of two technology tools: the Family Financial Tracker component of the Efforts to Outcomes case management and tracking system (EtO) and the EconomicCheckUp (ECU), a self-assessment tool, both of which are being piloted by the local community organizations participating as Economic Security Service Centers in ESI.
- NCOA Benefits Access: Evaluation of the Benefits Access Systems Change Initiative. This project uses a difference-in-difference approach based on pooled cross-sectional regressions to estimate the impact of the NCOA Benefits Access Initiative on senior enrollments in MSPs, LIS, and SNAP. The process study compares and contrasts the activities undertaken in the pilot sites to help explain variations in the findings from the quantitative analysis of project impacts and participants served. The process study also aims to identify effective practices that might be replicable in other sites.
Workforce Programs for Special Populations
- Indian and Native American Program. The purpose of this project is to help Indian and Native America Program (INAP) grantees achieve high program performance by improving the understanding and capacity of grantees and their staff in tracking and analyzing program performance and addressing low performance. As part of this project SPR is providing assistance in the area of web technology, capacity building in a range of areas of program operations, and training on performance measures as well as management information systems. See our online community of practice: https://ina.workforce3one.org/
- Migrant and Seasonal Farmworkers Program. This project, conducted under subcontract, combines the analysis of qualitative and quantitative data to assess the Migrant and Seasonal Farmworker (MSFW) program of JPTA and evaluates its effectiveness in meeting the needs of the hard-to-serve farmworker population. Specific objectives of this study are to assess the quality of the services being provided and determine whether these services met the needs of program participants, and to analyze the influence of federal policies and regulations on grantee operations.
- Strengths-Based Customer Service Strategies Technical Assistance and Tools. This technical assistance and training project sought to maximize ETA’s efforts to empower One-Stop Career Center frontline staff members in meeting the needs of underserved populations such as Indians and Native Americans, older workers, migrant and seasonal farmworkers, and persons with disabilities. Frontline staff learn strengths-based strategies to serving customers through various resources created by SPR, including webinars, a toolkit, a video, and in-person trainings. See the project website: https://doors.workforce3one.org/
- Workforce Opportunity Tax Credit (WOTC) TA Initiative. Sponsored by ETA, this technical assistance effort to assist state workforce agencies and employers with implementation of the new WOTC veteran provisions and overall effective program operation to enable better processing of WOTC veteran and target group certifications. See the programmatic and marketing materials for state workforce agencies, including the Employer’s Guide to WOTC.
For more information, or to discuss potential projects, please contact:
Director of the Workforce Development Division