Reflections on SPR’s Approach to Site Visits

This blog post was written by Madeleine Levin and Laura Pryor of Social Policy Research Associates.

“Conducting and Using Evaluative Site Visits”, a recent volume of New Directions for Evaluation (Number 156, Winter 2017) provides stimulating reading for evaluators. As SPR staff discussed this volume in our monthly qualitative methods brown bag, we were particularly interested in what our site visit approach could add to the discussion. Reflecting on the articles in this volume helped us articulate additional key features of SPR’s approach to evaluation site visits.  The purpose for most of these visits falls in the range of descriptive to developmental[1] to formative.  While we are often describing the implementation of a program—how closely it aligned with the planned model, and what challenges and successes staff and participants experienced during program implementation—we are careful to frame our visits as learning opportunities as opposed to monitoring or accountability visits.

As we reflected back on the thousands of site visits SPR staff have conducted, we concluded that the common denominator is listening, documenting, and finding the nuances and their drivers in program implementation.  Rather than looking for “compliance” or conformity to a program model, we are looking for the subtle variation in implementation that is often driven by local circumstances, pre-existing organizational structures or charismatic leadership.  Two programs can be implementing an intervention with high fidelity to an overall model, yet with variation that reflects how that model is adapted to that unique situation.

By offering an open, non-judgmental face when we are conducting site visits, we allow ourselves to consider local conditions and differences in local culture or community needs.  We can reflect with our hosts on what they have accomplished and how it differed from or fulfilled their original goals and expectations.  This approach is an important contribution to the discussion around conducting and using evaluative site visits. Furthermore, we take seriously the Framework for Site-Visit Standards, which is framed in the final chapter, and are committed to “taking a stand on standards” and ensuring that our practice is standards-based, an ongoing process in our research and evaluation practice.

[1] For a description of developmental evaluation, please see: Patton, M. Q. (2010). Developmental evaluation: Applying complexity concepts to enhance innovation and use. Guilford Press.