Administrative data – in other words, big data produced by public agencies and programs – is a valuable tool for program evaluation, research and analysis that can help improve government performance and tackle our nation’s pressing challenges. How can we enable more qualified researchers and government program managers to security and more easily access those data? We get insights from Dr. Amy O’Hara (@amy__ohara), a national expert on improving secure, responsible data access for research and evaluation.

Dr. O’Hara is a Research Professor in the Massive Data Institute and Executive Director of the Federal Statistical Research Data Center at the McCourt School for Public Policy at Georgetown University. She was previously a senior executive at the U.S. Census Bureau where she founded the Center for Administrative Records Research and Applications (CARRA), the Bureau’s administrative data unit.

What insights can public leaders gain from leading companies about building a culture of experimentation? To find out, we’re joined by Stefan Thomke, the author of the new book Experimentation Works: The Surprising Power of Business Experiments. Dr. Thomke is the William Barclay Harding Professor of Business Administration at Harvard Business School and has published widely on the topic of the management of innovation.

As background to our interview, a growing number government agencies and jurisdictions are taking an experimental approach to driving better results. For example, in making operational improvements to programs, they’re choosing an approach or strategy by rigorously testing different approaches and seeing which one works best. (That’s in contrast to choosing an approach based on best guesses or anecdotes or the opinion of the most senior person in the room.) For examples at the federal level, for instance, see those catalogued by the Office of Evaluation Sciences. An experimental mindset, however, is still nascent in most of government. Looking to leading examples and lessons learned from public and private organizations–including the companies studied by Dr. Thomke–can help more government leaders build an experimental culture.

Attracting a police force from diverse backgrounds is one important step in ensuring that citizens are well served by their police departments and have trust in the police. But what steps can police departments take that are effective in achieving that goal?

The Behavioral Insights Team, in conjunction with the What Works Cities Initiative, set out to work with more than 20 police departments to build credible evidence on that topic, including running randomized trials to test out different recruiting messages and other steps. The results are described is the report Behavioral Insights for Building the Police Force of Tomorrow.

To learn more, we are joined by Dr. Elizabeth Linos (@ElizabethLinos) who was one of the authors of the report. She is a professor of public policy at University of California, Berkeley, and is the former vice president and head of research and evaluation at the Behavioral Insights Team in North America.

Over the last two years, the Small Business Administration has made laudable progress in building and using evidence in order to learn what works and help programs improve. That includes launching an evaluation office and a chief data officer role within the CFO’s office, as well as creating a learning agenda to identify priority research questions from its bureaus.

To learn more, we are joined by Jason Bossie, who serves as the Director of SBA’s Office of Performance Management, serving under CFO Tim Gribben.

Web extra: Jason discusses the SBA’s acquisition vehicle for program evaluation and why it’s been useful. [click here]

It has been estimated that more than half of Americans are saving too little to support an adequate lifestyle if they plan to retire at age 65. It was economist and recent Nobel prize winner Richard Thaler who suggested a fix: Make payroll retirement savings plans available to everyone and then by add design features to make it easier for workers to make good choices.

The State of Oregon was the first out of the gate to do that. In 2017, it launched OregonSaves, a savings plan that covers private sector workers who do not otherwise have access to a savings plan in their workplace. Eight states have similar programs in the works, including California and Illinois, which are expected to start their versions in 2018. To learn more about OregonSaves, we are joined by its founding executive director, Lisa Massena.

Low-income and first-generation students enroll in and complete college at much lower rates than their more advantaged peers. This is particularly problematic because of the strong link between educational attainment and subsequent earnings, underscoring the need to find effective strategies that promote persistence and degree attainment.

We profile two such programs that are making an important difference, as shown by rigorous program evaluations. They are the City University of New York’s (CUNY’s) Accelerated Study in Associate Programs (ASAP) and the Dell Scholars program. We are joined by two researchers who helped lead the respective evaluations of these programs. Lindsay Page (@linzcpage) is a professor at the School of Education at the University of Pittsburgh and Michael Weiss (@MDRC_News) is a Senior Associate at MDRC.

Additional resources: Learn more by accessing MDRC’s reports on CUNY ASAP [click here] and the study of the Dell Scholars Program by Lindsay Page and her co-authors, Stacy Kehoe, Benjamin Castleman and Gumilang Sahadewo [click here].

The U.S. Department of Health and Human Services recently launched a pilot version of a new initiative called the Data Science CoLab, an eight-week-long data science training program. The first class, which kicked off in October 2017, included 25 employees from different agencies within HHS and with different levels of knowledge about using data. Participants applied to work on a specific data project that they proposed. The initiative builds on related HHS innovation initiatives, including the IDEA Lab and the Ignite Accelerator.

To learn more, we are joined by Will Yang. He has been an Innovation and Design Consultant at HHS for more than five years and now leads the CoLab.

Additional resource: A recent FedScoop article discusses Will Yang’s reflections on the first cohort of the CoLab [click here].

Related interview: An earlier Gov Innovator podcast interview with Bryan Sivak, then CTO of HHS, discusses implementing a department-wide innovation strategy [click here].

The Bipartisan Budget Act of 2018, passed by Congress and signed by the President on February 9th, 2018, contains several noteworthy bright spots in the use of data and evidence-based policy. Those wins suggest there is continuing bipartisan support in Washington for using evidence, data and innovation to improve the results and cost-effectiveness of Federal programs and policies.

To walk us through the most notable examples, we are joined by Nick Hart (@NickRHart), the director of the Evidence-Based Policymaking Initiative at the Bipartisan Policy Center (BPC).

Additional resources: For more information, see overviews by BPC [click here], Results for America [click here] and three posts from Feb.  9th from the Social Innovation Research Center [click here].

A small but growing number of federal departments and agencies have created  evaluation policies that describe the principles that those agencies seek to promote when they conduct program evaluations. Those principles can include rigor, relevance, transparency, independence, and ethics.

To learn more about evaluation policies and why they are useful to federal agencies, we are joined by Naomi Goldstein, the Deputy Assistant Secretary within the Office of Planning, Research, and Evaluation (OPRE) at the Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services; and Molly Irwin, the Chief Evaluation Officer at the U.S. Department of Labor.

Additional resources: ACF’s evaluation policy is available on the OPRE website [click here]. DOL’s evaluation policy is available on the department’s Chief Evaluation Office website.[click here].

How can program managers within public agencies — whether local, state or federal — use random assignment (in other words, a lottery) within programs to build evidence that can strengthen results and improve customer service?

A good example comes from South Carolina. Its Medicaid program is administered through Managed Care Organizations, which offer different health care plans to Medicaid beneficiaries. What happens when people don’t choose a plan? In those cases, the state has begun randomly assigning those individuals to plans. It’s not only a fair way to make those assignment decisions, but it also allows researchers to build credible evidence about the different Medicaid plans — and about the state’s star ratings of those plans.

To learn more, we are joined by Matthew Notowidigdo, and an affiliated professor at J-PAL. With Craig Garthwaite of Northwestern, he has been studying South Carolina’s Medicaid system as part of JPAL-North America’s State and Local Innovation Initiative.