Becoming an evidence focused grant-making organization: An interview with Kelly Fitzsimmons, Vice President, Edna McConnell Clark Foundation

How can grant making agencies in the public sector build their capacity to build and use rigorous evidence and help their grantees do so too? We gain insights from a successful evidence focused grant-making organization outside of government, the Edna McConnell Clark Foundation.

A hallmark of the Foundation’s approach is a focus on evidence. It chooses and structures its investments largely on the basis of empirical evidence that a grantee or potential grantee’s programs help economically disadvantaged young people. And a major objective of the Foundation’s investments is to help grantees build their own evidence base. In doing this work, it uses a framework to assesses an organization’s evidence of effectiveness on a continuum from high apparent effectiveness to demonstrated effectiveness to proven effectiveness.

To learn more, we’re joined by Kelly Fitzsimmons who is the Foundation’s Vice President and Chief Program and Strategy Officer.

Web extras: Kelly Fitzsimmons discusses:

  • The Foundation’s pilot program PropelNext, designed to help grantees in early stages of evidence building to systematically collect and analyze data [click here]
  • The growing emphasis among nonprofits around rigorous evidence and evaluation [click here]
  • Her advice to organizations just starting their journeys to become more evidence focused [click here]

Additional resources: To learn more about tiered-evidence grant programs, in which larger grant dollars go to approaches backed by stronger evidence (a.k.a. innovation funds), see the video tutorial on the blog. Also, to see another example of a framework to assess evidence levels, see the regulations for the Investing in Innovation (i3) program at the Department of Education, page 18683, which discuss criteria for development, validation and scale-up grants.

Share Button

How program managers can use low-cost experiments to improve results: A video overview

How can public leaders and program managers use low-cost experiments — also knows as low-cost randomized controlled trials (RCTs) — to improve program results in government? In this blog post, rather than conducting an interview as I usually do, I provide a video overview of the topic and provide examples. An audio version is also available, above.

As I explain in the video, low-cost experiments use existing high-quality data that are already being collected, which can bring the cost of rigorous evaluation way down. As a result, low-cost experiments can be a valuable complement to more traditional evaluation approaches and open up more opportunities for program managers to experiment and learn what works.

Additional resource: An interview on the blog with Jon Baron of the Coalition for Evidence-Based Policy on “Rigorous program evaluation on a budget,” highlighting more examples of low-cost experiments, is located here.

Share Button

Implementing a department-wide innovation strategy: An interview with Bryan Sivak, Chief Technology Officer, U.S. Department of Health & Human Services

How can public agencies at the federal, state or local levels spur innovation to tackle tough problems and find ways to better achieve their missions? To gain insights, we’re joined by Bryan Sivak (@BryanSivak), the Chief Technology Officer at the U.S. Department of Health and Human Services (HHS). Under his leadership, HHS launched its IDEA Lab in 2013, which has already catalyzed more than 100 innovation projects. Prior to his current role, he was the Chief Innovation Officer to Maryland Governor Martin O’Malley, Chief Technology Officer for the District of Columbia and a technology entrepreneur in the private sector.

As Bryan explains, the three main strategies of the HHS IDEA Lab are:

  • Supporting innovators from within the department, e.g., the HHS Innovates initiative that identifies and celebrates internal innovation by employees
  • Bringing new ideas and concepts into the department, e.g, the HHS entrepreneurs and HHS innovators-in-residence initiatives that bring in innovators from outside the department to help tackle important challenges
  • Mobilizing communities of practice to work on discrete challenges or ongoing, cross-cutting initiatives that require creative thinking and new solutions, e.g., the HHS Health Data Initiative

Bryan also provides broader advice for public leaders who want to strengthening a culture of innovation in their agencies. The clip of this portion of the interview is available here.

Share Button

Six ways government can use incentive prizes to spur innovation: An interview with Jesse Goldhammer, Principal, Deloitte Consulting

Incentive prizes — also known as prize competitions or challenges — are increasingly being used to spur innovation and address key challenges by public agencies at the federal, state and local levels. A recent report published by Deloitte Consulting, The Craft of Incentive Prize Design: Lessons from the Public Sector, provides insights and advice, including discussing six main outcomes, or goals, that different incentive prizes are designed to address: 1) attract new ideas; 2) build prototypes and launch pilots; 3) stimulate markets; 4) raise awareness; 5) mobilize action; and 6) inspire transformation.

Joining us to discuss the report is one of its co-authors, Jesse Goldhammer. He is a Principal with Deloitte Consulting.

Web extras: Jesse Goldhammer discusses how incentive prizes are used to mobilize action [click here] and inspire transformation [click here]. He also provides advice about crafting incentive prizes [click here].

Additional resource: In a related Gov Innovator interview, Jenn Gustetic of NASA discusses that agency’s use of prizes and challenges [click here].

Share Button

Boosting the Life Chances of Young Men of Color: An interview with Dan Bloom, Director of Health and Barriers to Employment Policy Area, MDRC

Despite progress in many areas, young men of color still face many obstacles to success in terms of education, employment and other areas. Today, there is growing momentum through government and other efforts to improve outcomes for young men of color, including New York City’s Young Men’s initiative and the Obama Administration’s My Brother’s Keeper initiative. A recent report by the social policy research firm MDRC titled Boosting the Life Chances of Young Men of Color reviews what we know about interventions to improve the outcomes of young men of color that have been shown to be effective through rigorous research.

To discuss the report’s findings, we’re joined by Dan Bloom who, with Christopher Wimer, authored the report. Dan is the Director of the Health and Barriers to Employment Policy Area at MDRC.

Web extra: Dan Bloom discusses a promising area for future rigorous evaluation: strategies for exposing disadvantaged high school students to the labor market [click here]

Share Button

Using logic models, a key building block of results-focused programs: An interview with Tom Chapel, Chief Evaluation Officer, Centers for Disease Control and Prevention

Just like mapping out a journey before embarking on a trip, logic models provide a type of map for programs about where they want to go and how they plan to get there. To learn more about logic models and how they can be useful to programs and public managers, we’re joined by an expert on the topic, Tom Chapel. He’s the Chief Evaluation Officer at the Centers for Disease Control and Prevention (CDC).

The interview provides an overview of:

  • Why a clear program description is important
  • What logic models are and how they’re used
  • How the CDC is using logic models to clarify grantee proposals
  • Advice to program leaders and public managers about using logic models

Web extra: Tom Chapel discusses some additional key terms often used in logic models. [click here]

Share Button

Using predictive analytics and rapid-cycle evaluation to improve program design and results: An interview with Scott Cody, Vice President, Mathematica Policy Research

What are predictive analytics and rapid-cycle evaluation and how can public agencies and programs use them to improve program delivery and outcomes? To explore these questions, we’re joined by Scott Cody. He’s a Vice President of Mathematica Policy Research and the co-author, with Andrew Asher, of a recent paper “Smarter, Better, Faster: The Potential for Predictive Analytics and Rapid-Cycle Evaluation to Improve Program Development and Outcomes,” published by the Hamilton Project at the Brookings Institution.

Web extra: Scott Cody provides two suggested steps for public agencies that want to strengthen their ability to use predictive analytics and rapid cycle evaluation. [click here]

Share Button

Why successful performance measurement starts with considering purpose: An interview with Bob Behn, Professor, Harvard Kennedy School

Why measure performance? As Bob Behn of the Harvard Kennedy School notes, performance measurement is not an end in itself. Instead, it can be helpful in achieving specific managerial purposes such as to evaluate, control, budget, motivate, promote, celebrate, learn and improve. As a result, public managers need to think seriously about why they want to measure performance before choosing what to measure.

To learn more, we’re joined by Bob Behn. His article “Why Measure Performance? Different Purposes Require Different Measures” (also summarized here) was recently named one of the top 75 most influential articles by the journal Public Administration Review. He’s a lecturer at the Harvard Kennedy School and the faculty chair of the executive education program Driving Government Performance. He’s also the author of the new book, The PerformanceStat Potential.

Share Button

Children’s executive functions and evidence-based activities that improve them: An interview with Adele Diamond, Professor, University of British Columbia, Vancouver

 Executive functions (EFs) are a critical factor in determining people’s success in school and on the job, as well as their mental and physical health. Moreover, the core executive functions (self-control, working memory and cognitive flexibility) form the foundation for higher-order executive functions, such as reasoning, problem solving, and planning. Importantly, research shows that executive functions can be improved, including for young children, which can bring a lifetime of benefits.

To learn more, we’re joined by one of the leading experts, Adele Diamond (@DrAdeleDiamond). She’s a Professor of Developmental Cognitive Neuroscience at the University of British Columbia, Vancouver, and one of the founders of the field of Developmental Cognitive Neuroscience. She earned her B.A. from Swarthmore College and her Ph.D. from Harvard.

Because this interview is longer (30 min) than most, here are links to specific segments if you prefer:

  • Why inhibitory control is important, click here.
  • Why working memory is important, click here.
  • Why cognitive flexibility is important, click here.
  • Strategies to strengthen exec functions in children, click here.
  • Implications of the research for education policy and practice, click here.

Further reading:

  • Dr. Diamond’s survey article on activities and programs that improve children’s executive functions.
  • study (referenced in the interview) by Terrie Moffit et. al that draws on data from over 1,000 children born in one city in a single year and tracked until age 32.
Share Button

Avoiding performance perversity: An interview with Donald Moynihan, Professor, University of Wisconsin-Madison

An increasing number of real-life examples have shown the potential for performance measures to create perverse behavior within public agencies and programs. How can public leaders and managers structure performance measures and incentives, and take action, to try to avoid so-called performance perversity?

To gain insights, we’re joined by Donald Moynihan. He’s a Professor at the La Follette School of Public Affairs at the University of Wisconsin-Madison and the author of the book The Dynamics of Performance Management. He also authored a recent LA Times opinion piece on the problem of performance perversity within public agencies.

Share Button