The new federal education law, the Every Student Succeeds Act (ESSA), emphasizes the importance of evidence, including defining four levels of evidence-based practices. The law, however, leaves it to states to decide how much they want to build an evidence base and how much to nudge districts toward choosing more effective strategies. So what should state education leaders do who want to leverage the new law and encourage districts to learn and do what works for students?

Tom Kane joins us for a two-part series to provide suggestions. In this podcast episode, he discusses how states could use the authority and resources provided by ESSA to launch a system of “efficacy networks,” meaning collections of local agencies committed to measuring the impact of the interventions they’re using. As he notes, “An overlapping system of efficacy networks working with local [education] agencies would create a mechanism for continuous testing and improvement in U.S. education. More than any single policy initiative or program, such a system would be a worthwhile legacy for any state leader.”

He also describes how the Proving Ground initiative run by the Center for Education Policy Research (CEPR) at Harvard is demonstrating the value of having an efficacy network. CEPR is working with 13 school agencies to develop a model to easily conduct low-cost, local pilots.

Tom Kane is a professor of education and economics at the Harvard Graduate School of Education and faculty director of CEPR. His recent article in the journal Education Next is called, “Making Evidence Locally: Rethinking education research under the Every Student Succeeds Act.”

“Many of the most important functions of state and local governments – from building and maintaining roads to housing the homeless – involve contracting for goods and services supplied by the private sector,” notes the Harvard Kennedy School’s Government Performance Lab in their primer, Results-Driven Contracting: An Overview. “Unfortunately, governments often treat procurement as a back office administrative function, rather than as a core part of their strategy for delivering better performance.”

As a result, increasing the effectiveness of procurements offers an enormous opportunity today for state and local governments to improve their overall performance. As part of Bloomberg Philanthropies’ What Works Cities initiative, the Government Performance Lab has been working with a set of city governments to adopt results-driven contracting strategies and to transform the culture of procurement.

To learn more, we are joined by the Lab’s director, Jeffrey Liebman. He is a professor at the Kennedy School and is also the Director of the Taubman Center for State and Local Government. His recent article published on the website Route Fifty is, “Business as Usual Can Be the Riskiest Procurement Approach.”

Almost all Federal agencies are lead by a Secretary and a Deputy Secretary. But in 2000, Congress created a new position at the State Department, the Deputy Secretary of State for Management and Resources — in other words, the chief operating officer. In doing so, the Department became the only federal Cabinet-level agency with two co-equal Deputy Secretaries.

What lessons does having a second Deputy Secretary provide for other public agencies, whether federal, state or local? To get insights, we speak with Heather Higginbottom (@hhigginbottom). She served as the Deputy Secretary of State for Management and Resources from 2013 to the end of the Obama administration. Before that role she served as Deputy Director of the White House Office of Management and Budget. She currently serves as the Chief Operating Officer of CARE, the global humanitarian organization. We recorded this conversation in 2016 while she was still in her Deputy Secretary role.

Update: The Trump administration appears to be combining the two Deputy Secretary roles at State into one, reversing the structure during the Obama administration.

How can Federal agencies successfully streamline their support services, such as HR and IT, to boost efficiency and improve results?

We get insights from Jeffrey Neal (@JeffNealHR), a leading expert in human resources issues. He served for 33 years in the Federal government, including as the Chief Human Capital Officer for the 23,000-employee Defense Logistics Agency (DLA) from 2000 to 2009. At DLA, he helped the agency identify more than $50 million in administrative and operational savings. He later served as the Chief Human Capital Officer for the Department of Homeland Security, which has more than 240,000 civilian and military employees. Today he is a Senior Vice President at the consulting firm ICF and he also runs the ChiefHRO blog.

What is the value of evidence and data for elected city leaders as well as how can those leaders create a results-focused culture within city government? We get insights from Michael Nutter who served for eight years at the Mayor of Philadelphia, from 2008 to January 2016. Under his leadership, Philadelphia became known as a leader in the use of data and evidence.

In particular, the Nutter Administration established strategic goals with measurable targets; launched PhillyStat, Philadelphia’s performance management system; established Philadelphia’s open data policy in 2012 and launched an open data portal in 2015; and launched Philly 311, the city’s online customer service system.

Today Michael Nutter is a CNN political commentator, a professor at Columbia University, a fellow at the University of Chicago’s Institute of Politics and a senior fellow with the What Works Cities initiative, among other roles.

Public leaders — whether they’re helping run a state agency, a school system, a hospital, a set of Head Start centers or any other organization — are likely to implement changes over time, whether it’s adjusting programs or adding new services. Maybe it’s a new curriculum for students in a school district or new intake procedure for patients in a hospital. Whatever the change, how can those leaders determine if the change is actually effective?

Our focus today is new software, called RCT-YES, designed to help public leaders (and the researchers who work with them) answer that question. It was funded by the Institute of Education Sciences, the statistics, research, and evaluation arm of the U.S. Department of Education, and developed in partnership with Mathematica Policy Research. The software, available free to download online, is based on new statistical methods for analyzing data from randomized controlled trials (RCTs).

To learn more, we are joined by Peter Schochet. He is a nationally known methodological expert in program evaluation and a Senior Fellow at Mathematica. He led the team that developed RCT-YES.

Web extra: For those with deeper expertise in evaluation, Peter Schochet gives an overview of how the RCT-YES software is designed to conduct a wide range of analyses using RCT or QED data and how the software uses new statistical methods for analyzing those data. [click here]

In 2010, the Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services launched a project to explore how programs could advance their goals, and address specific challenges, by applying insights from behavioral sciences, including behavioral economics. It is called the Behavioral Interventions to Advance Self-Sufficiency (BIAS) project. Now, six years later, it has results from 15 randomized experiments conducted across seven states on the topics of employment, child support and childcare.

To get an overview and hear implementation lessons for human services agencies that might want to use these types of interventions — or “nudges,” as they are often called — we are joined by two researchers from the social policy research firm MDRC, which was a partner on the BIAS project. Lashawn Richburg-Hayes is a Director and Nadine Deshausay is a Research Associate at MDRC.

More information: For more information on the 15 projects, including their goals, strategies, results and costs, see MDRC’s PowerPoint presentation presented at the BIAS Capstone Convening in April 2016 [click here].

Programs and agencies in government often exist in silos, where the efforts of one aren’t necessarily connected with others and their data are not shared between them. That slows the process within government of learning what works, coordinating efforts, spurring social innovation, and continuous improvement.

A growing number of states and localities, however, are developing Integrated Data Systems by linking their program data, also called administrative data, across multiple agencies to monitor and track how services are being used and to what effect. These systems can also be used to test social policy innovations through quick, low-cost randomized control trials and quasi-experiments, as well as to do continuous quality improvement efforts and benefit cost analysis.

To learn more, we’re joined by Dennis Culhane. He’s is a professor in the School of Social Policy and Practice at the University of Pennsylvania and the co-principal investigator of the nonprofit Actionable Intelligence for Social Policy (AISP) which has built a network of jurisdictions using Integrated Data Systems.

Credit: The graphic above is from AISP’s overview of Integrated Data Systems.

Federal programs produce a lot of data — known as administrative data — and those data can be very useful for program administrators and researchers to answer important questions about policy and practice. That is especially true when data from multiple programs or datasets are linked, producing a broader view of program performance that spans organizational silos.

In short, access to administrative data is critical to making Federal programs and policies more effective and efficient. Today, however, access to data can be so restricted that conducting research and analysis can be very difficult.

Our guest today has a vision for how that could be different and why greater access to data is important. Maria Cancian is a Professor of Public Affairs and Social Work at the University of Wisconsin–Madison and the former Director of the Institute for Research on Poverty. From 2015 to 2016, she served as the Deputy Assistant Secretary for Policy at the Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services.

Additional resource: The White House Office of Management and Budget recently created a set of background briefs on using administrative data, prepared for the Commission on Evidence Based Policymaking [click here].

How can public agencies can use rapid, low-cost experiments to test (and learn from) low-cost, light-touch interventions such as communications and outreach strategies? Also, how can agencies partner with academic researchers to run those experiments and what characteristics of those researcher-practitioner partnerships help make them successful?

To get insights into all those topics, we are joined by Adam Sacarny (@asacarny). An economist by training, he is a professor of health policy and management at Mailman School of Public Health at Columbia University. He has been working with both the U.S. Department of Health and Human Services (HHS) as well as the State of Colorado on communications-related experiments. Both HHS and Colorado are taking a “test, learn and adapt” approach by testing out certain interventions using relatively quick, low cost randomized controlled trials, learning from the results, and refining the strategies to be tested again.

Web extra: Adam Sacarny discusses the communications-related experiment he conducted with the State of Colorado around helping Colorado citizens choose a health plan on the ACA marketplace that best fits their needs. [click here]