After the Evaluation

Available on-demand

Can a bus pass prevent a jail sentence? Listen in to discover the “myth-busting” power of co-discovery. Join King County, Notre Dame’s LEO, and Stanford’s REGG Lab as they explore how high-stakes partnerships transform rigorous econometrics into equitable, real-world policy change.

Featured Speakers:

May Lynn Tan, Assistant Deputy Director, Evidence for Action (RWJF) (at time of study)
Matthew Freedman, Professor of Economics, University of California-Irvine
Daniel E. Ho, William Benjamin Scott and Luna M. Scott Professor of Law, Standford University
Kimberly Dodds, Homelessness Prevention Manager, King County
David Phillips, Associate Research Professor, LEO
Becky Elisa, MBA, Food Protection Program Environmental Health Services, Public Health- Seattle & King County (at time of study)
Dr. Eyob Mazengia, Assistant Division Director, EH Seattle & King County Public Health
Maria Jimenez-Zepeda, ORCA Reduced Fare Project Program Manager, King County Metro Transit

Partner with LEO

The Evidence Matters series is sponsored on ThinkND by the Wilson Sheehan Lab for Economic Opportunities (LEO). Poverty is stubborn and requires the utmost collaboration of thought and action to drive change. People of goodwill must bring their unique strengths and positions together to solve this problem. At LEO, we believe knowledge has to be combined with action. But poverty can’t be solved by just one person, or even one sector. That’s why we bring together innovative social service provider partners, top-tier academics, philanthropists, policymakers, and others to tackle poverty.

Change is possible. And with your action, we can get one step closer to reducing poverty in our country, together.

Your job is to act. What will you do?

For more information, please visit LEO’s website.

The “Evidence Matters” webinar series highlights a fundamental shift in how public policy is crafted and refined. Rather than viewing evidence as a static destination—a final report to be filed away—the collaboration between King County, Notre Dame’s Lab for Economic Opportunities (LEO), and Stanford’s REGG Lab treats research as a cycle of continuous learning. For the strategic policymaker, the value lies not just in the data itself, but in building an infrastructure for “co-discovery” that bridges the gap between academic rigor and community needs.

Lessons from the Field: Evaluating Key Interventions

Practical evaluations of social programs often yield complex results that demand nuanced interpretation. In the Court Appearance Transit Subsidy study, researchers addressed a critical issue: failure-to-appear (FTA) rates can reach 30% in some jurisdictions, leading to dire consequences like immediate arrest. To test if transportation was the primary hurdle, the team scaled subsidies from $15 to three months of free transit. The result was a “null effect” on court attendance.

However, this finding is a strategic win, not a failure. It allowed the county to rule out transportation as the primary causal barrier, paving the way for multifaceted interventions like text-message reminders. Simultaneously, the Fair Free Transit Study proved that while subsidies didn’t fix court attendance, they increased overall transit usage by two to three times, highlighting the profound mobility needs of low-income residents. Similarly, the Youth and Family Homelessness Prevention Initiative demonstrated that while simple financial aid suffices for some, others require intensive case management to successfully avoid eviction. By differentiating these needs, the county can allocate scarce resources with surgical precision.

Shifting Organizational Culture: The Food Safety Example

The partnership between King County Public Health and Stanford’s REGG Lab illustrates how research can professionalize agency culture. Before the intervention, inspector results were often “close to random” due to high inter-inspector variability, threatening the consistency of legal administration. By implementing a “peer review” program where inspectors jointly visited facilities to calibrate findings, the agency moved from superficial data collection to a culture of evidence-based decision-making. This shift empowered frontline staff to move beyond reactive posture, transforming the agency into an environment where the standard inquiry is: “What does the data say?”

The Mechanics of Successful Collaboration

Navigating the inherent tensions between academia and policy requires deliberate management. Academic researchers typically seek the “perfect” answer on an infinite timeline, while practitioners require “good enough” data for timely action. Bridging this gap requires “barrier busters” like Carrie Chihawk, who navigate the gritty logistical hurdles of policy work—such as an antiquated Washington law that essentially restricted data access by charging $10 per cassette tape for court recordings.

Success is built on mutual trust, allowing for real-time adjustments, such as increasing subsidy amounts mid-study when data showed defendants were exhausting credits faster than anticipated. These sustained partnerships move beyond one-off evaluations to create a community of practice that views the vulnerability of learning as a prerequisite for progress. By embracing this collaborative rigor, King County and its academic partners are building a more equitable framework for the communities they serve.

The Diagnostic Power of Null Results: Findings that show “no effect” are critical diagnostic tools. They allow practitioners to rule out ineffective levers—such as transportation for court attendance—and pivot resources toward more complex causal barriers.
The “Co-Discovery” Model: The most effective partnerships avoid the “hammer looking for a nail” approach. Instead, they utilize an exploratory process where researchers and practitioners jointly identify unanswered questions and ground-truth theories through open communication.
Investing in the “Runway”: Successful partnerships require significant resourcing before a grant is even applied for. Dedicated “runway” time for trust-building and data-sharing negotiations is essential for long-term project viability.
Dissolving the Research Hierarchy: The framework recognizes that the distinction between “Capital R” (academic) and “little r” (practitioner) researchers is false. On-the-ground practitioners are researchers in their own right, providing essential context that prevents methodological errors.
Institutionalizing “Barrier Busters”: Sustaining evidence-based practice requires internal agency roles dedicated to “barrier busting.” These individuals streamline data subpoenas, manage general counsel negotiations, and ensure that research findings actually influence decision-making.

“There’s no shame in knowing that we’ve been doing it wrong. I think what would be unfortunate is to continue to do it that way without actually evaluating our work.” — Dr. Eyob Mazengia
“The best partnerships are really a process of co-discovery… It’s much more effective if you have a kind of open line of communication to explore the range of different projects you might be able to tackle together.” — Daniel E. Ho
“That distinction that I had made in my head between capital R and little r was not true and it was not accurate… those expertises are from all of the researchers or each practitioner; they’re all contributing to this bigger learning.” — Maria Jimenez-Zepeda
“The academic world wants a perfect answer on an infinite timeline… having a good answer on the right timeline is sometimes more important than having a perfect answer on the wrong timeline.” — David Phillips