Statistics New Zealand (@Stats_NZ) is the government department of New Zealand charged with collecting and producing statistical information. It is known as a leader in terms of linking data from different data sets in order to enable research and insights into important and complex policy challenges with the goal of improving outcomes for New Zealanders. For over ten years, Stats NZ (as it is also called) has been working on data integration including the creation of the Integrated Data Infrastructure (IDI) in 2011. Today the IDI is is a large research database containing microdata from a range of government agencies, including over 165 billion facts.

To learn more, we are joined by Kelvin Watson, the Deputy Chief Executive for Data Services at Stats NZ.

The Department of Human Services (DHS) in Allegheny County, which includes Pittsburgh, is a leader in the use of data to continually improve services for its residents. In 1999, DHS created its Data Warehouse that consolidated its human services data relating to topics such as behavioral health, child welfare and homeless services. It then expanded this database to include data from other county agencies as well as the Pittsburgh Public Schools.

How was the department able to link data across programs and with providers, given what often seems like insurmountable privacy laws that can make data sharing difficult? We get insights from Erin Dalton, the Deputy Director for the Office of Data Analysis, Research and Evaluation (DARE), and Brian Bell, a supervisor within DARE and also the privacy officer at the department.

Our interview builds on an earlier conversation with Erin Dalton that provides an overview of the Data Warehouse.

What principles can help guide public leaders—whether policymakers or public managers—in their use of evidence-based policy to improve results? Howard White (@HowardNWhite) of the Campbell Collaboration joins us to share four fundamental principles:

1. Use the right evidence to answer the right question. Different types of evidence — e.g., monitoring, process evaluation, impact evaluation and systematic reviews — all can produce useful information for decision makers. But each type of evidence should not be used to answer questions that are beyond its usefulness.
2. Don’t rely on single studies. When possible, leaders should avoid making important funding decisions based on single studies, especially those done in one site. That’s because the findings from one study are often different from those of further studies. The best approach is to use systematic reviews (where they exist), meaning syntheses of multiple high-quality studies.
3. Context matters for transferring evidence. Why do findings from one study often not replicate in another? A key reason is that context matters. For example, when a home visiting program found to be effective in the U.S. was tested in Britain, it produced no impact. Why? Likely it was the different context: Britain already provides services to low-income parents that are quite similar to the home visiting program in the U.S. It is why leaders should test out, with rigorous evaluation, programs and initiatives in their own setting, particularly if previous research was conducted in a different context.
4. Evidence-based policy is not a blueprint (aka cookie cutter) approach. This is a way of summarizing the previous two principles. While architects can take the blueprints for one building and build the same building elsewhere, and chefs can take a recipe from one restaurant and cook it in another, public leaders need to be careful when applying research from one place or setting to another.

Dr. White is the Chief Executive Officer of the Campbell Collaboration, a nonprofit best known for its use of systematic reviews to help policymakers and others make well-informed decisions. Previously he was the founding Executive Director of the International Initiative for Impact Evaluation (3ie) and led the impact evaluation program at the World Bank’s Independent Evaluation Group.

Photo credit: European Union

Last month, President Trump issued an executive order calling for agencies to analyze their structures and programs in preparation for a major reorganization. The Trump administration’s focus on reorganization raises the question of when is reorganization useful and what pitfalls need be avoided.

To get insights, we’re joined by Bob Behn. Although it is too early to comment on the Trump administration’s reorganization in particular, since it is still being formulated, he draws on lessons from other reorg efforts to provide advice to public leaders.

Bob Behn is one of the nation’s leading experts on leadership and performance management in government. He is a lecturer at the Harvard Kennedy School and the faculty chair of its executive program called Driving Government Performance. He also publishes a free monthly newsletter called Bob Behn’s Public Leadership Report.

One of the most innovative new approaches in Federal policy is known as Performance Partnership Pilots, also called P3. In 2014, Congress authorized several Federal agencies, including Education, Labor and HHS, to enter into up to ten Performance Partnership agreements per year with states, regions, localities, or tribal communities. These agreements allow these jurisdictions to have additional flexibility in using discretionary funds across multiple Federal programs, in exchange for a commitment to achieve significant improvements for disconnected youth, meaning youth that are not in school and not working. The P3 concept, however, could in theory be applied to many other social policy areas as well.

To learn more, we are joined by Thaddeus Ferber, a Vice President at the Forum for Youth Investment, an organization that advocated for and helped bring about the P3 authorization.

In 2015, a unique collaboration was launched call the Rhode Island Innovative Policy Lab (RIIPL). It is a partnership between researchers at Brown University and the Office of the Governor of Rhode Island, with the goal of helping state agencies design evidence-based policies to better serve Rhode Island families.

RIIPL’s goal is to use data and science to improve policy, alleviate poverty and increase equity of opportunity. To do that work, it has created a new linked database of public programs, connecting more than 100 previously independent data sets.

To learn more and hear about some example projects, we are joined my Justine Hastings (@JHastings_Econ). She is an economist at Brown University and the founding director of RIIPL.

In this podcast episode, we explore two real-life stories from the front lines of government performance improvement efforts that highlight the importance of replication and validation in evidence-based policy.

In the U.K., the department of revenue and customs, in conjunction with the U.K. Behavioral Insights Team (BIT), tested new versions of the letter sent to people who were late in paying their taxes. The new versions simply added ones additional sentence, a behavioral “nudge” that drew on the power of social norms. As the New York Times explained:

One nudge was a sentence telling recipients that a majority of people in their community had already paid their taxes. Another said that most people who owe a similar amount of tax had paid. Both messages bolstered tax collection, and combining them had an even stronger effect. Over the last financial year, the letters brought forward £210 million of revenue, Britain’s revenue and customs department says — money that otherwise would have had to be chased in costly court procedures and failed to earn interest for the government.

A few years later, the U.S. Treasury Department, in conjunction with the U.S. Social and Behavioral Sciences Team (SBST) — the Obama Administration’s version of the BIT — decided to test a similar approach. They updated the letters sent to people who owed non-tax debt to the Federal government, simplifying and personalizing the letters and adding similar behavioral nudges as in the U.K.. The results showed no effect on payment rates, underscoring how context matters.

To learn more, we are joined by Tammy Chang, a Senior Economist at U.S. Department of the Treasury within the Bureau of the Fiscal Service, and Nathaniel Higgins, a co-founder and Fellow on the former SBST.

Massachusetts is known as a leader in providing education policymakers with research findings that they can use to improve policy and practices. The state’s Office of Planning and Research, within the Department of Elementary and Secondary Education (ESE), takes a multifaceted approach:

• Design a proactive research agenda, also known as a learning agenda
• Increase internal capacity for research
• Create partnerships with external researchers

To learn more, we are joined by Carrie Conaway (@clconaway), the Chief Strategy and Research Officer of the ESE. She leads the 15 person Office of Planning and Research.

Because many of the most important functions of state and local governments involve contracting for goods and services supplied by the private sector, improving procurement processes is an important way to strengthen outcomes for citizens. That is the motivation behind our series on results-driven contracting.

Our focus today is Seattle, Washington, whose Human Services Department worked with the Government Performance Lab (GPL) at the Harvard Kennedy School to improve outcomes for its homeless population. The changes included:

• Consolidating contracts and allowing for more flexibility
• Establishing goals for homeless service providers and tracking progress
• Using active contract management

To learn more, we are joined by is Jason Johnson, the Deputy Director of Human Services Department in Seattle.

Additional information: Check out the brief by the GPL about Seattle’s contracting reforms. [click here].

Forty-two states and the District of Columbia spent $6.2 billion in state funds on pre-kindergarten programs in 2015, highlighting the emphasis that policymakers are placing on pre-k to help students prepare for elementary school. Research has shown both the success of pre-K as well as inconclusive evidence about the sustainability of those gains as children become older. Those findings raise the question: How can states optimize their pre-K programs to provide both the strongest early learning boost and a solid foundation for future learning?

Recently, a group of leading pre-K researchers set out to find consensus about what we know about pre-K education. In April 2017 the Pre-Kindergarten Task Force presented their findings. To learn more, we are are joined by a member of the task force, Greg Duncan. He is a Distinguished Professor in the School of Education at the University of California, Irvine.

In the interview, Duncan summarizes the research evidence around four key decisions (policy levers) facing state policymakers related to pre-K: (1) Whether to fund more or fewer pre-K slots; (2) Whether and how to regulate classroom quality; (3) Whether and how to prescribe curriculum; and (4) How to support the gains of pre-K after children leave pre-K, i.e., in the elementary school years.