For public agencies at the federal, state or local levels that want to strengthen their evaluation capacity, creating a Chief Evaluation Officer role can be an important step. A leading example comes from the U.S. Department of Labor. The Chief Evaluation Officer position was created within the Department in 2010 in order to coordinate the Department’s evaluation agenda and work with the 17 agencies within the Department to design and implement evaluations. Today, the office has a staff of about ten people. It is led by Demetra Nightingale who has served as the Chief Evaluation Officer since 2011. The office is guided by a the Department of Labor’s Evaluation Policy, which emphasizes rigor, relevance, transparency, independence and ethics.

Demetra Nightingale join us to discuss the role that she and her office play. A researcher, evaluator and social policy expert, she was appointed as Chief Evaluation Officer after a 29 year career at the Urban Institute.

Web extra: Demetra Nightingale describes the new Data Analytics Unit within her office. [click here].

C-Stat is a strategic management strategy launched in 2012 by the Colorado Department of Human Services (CDHS) under Executive Director Reggie Bicha, serving under Governor John Hickenlooper. It is a tool that the Department uses to examine what’s working and what needs improvement. In particular, each of the five offices within the Department collects performance data on a regular basis and examines those data in monthly C-Stat meetings with the Executive Director and his executive team.

To learn more, we’re joined by Ki’i Powell who is the Performance Management Director at CDHS and oversees the day to day operations of C-Stat.

Web extras: Ki’i Powell talks about who is in the room for C-Stat meetings and what is the layout of the room. [click here]. (You can also see a diagram of the room layout here.) She also discusses a specific example of how C-Stat helped drive change on an important issue: increasing family contact with youth who are in correctional facilities. [click here]

Additional resource: In a related Gov Innovator interview, CDHS Executive Director Reggie Bicha speaks about KidStat, which he launched in his previous role as Secretary of the Department of Children and Families in Wisconsin. [click here]

The nonprofit Partnership for Public Service recently released a report, co-sponsored with Deloitte, titled Helping Government Deliver: Transforming Mission and Support Services. It describes how agencies and jurisdictions are using consolidated and shared services to improve efficiency, results and customer service. In particular, the report discusses how public leaders can move beyond the familiar model of sharing support functions for a single line of business (e.g., payroll or human resources to sharing multiple support and mission-critical functions within an entire department and, in some cases, across departments.

To learn more, we’re joined by the report’s project lead, Judy England-Joseph. As background to the interview, the report describes a continuum of shared services delivery that includes:

• Single line of business: Sharing a single support function across an agency, such as human capital, IT, financial management or accounting (what most agencies that share services do today)
• Multiple lines of business: Sharing multiple support functions across an agency (we discuss NASA’s Shared Services Center as an example)
• Intra-agency mission services: Sharing support and mission services across an agency (we discuss the Department of Energy’s Office of Environmental Management as an example)
• Interagency mission services: Sharing support and mission services across different agencies or jurisdictions (we discuss the City of Charlotte and Mecklenburg County as an example)

How can public leaders and program managers gain rigorous, useful insights into program effectiveness at a modest cost? A report by the Coalition for Evidence-Based Policy, “Rigorous Program Evaluations on a Budget,” offers suggestions.

As the report discusses, low-cost, high quality impact evaluation involves two key elements. The first is building random assignment evaluations into policies (e.g., using a lottery if there aren’t funds to serve everyone to create program and control groups). The second is using existing administrative data (data that a program is already producing) on outcomes of interest, so you can avoid the cost of collecting new data.

To learn more, we’re joined by Jon Baron, the President of the Coalition for Evidence-Based Policy. Our interview includes an overview of three programs evaluated with low-cost randomized controlled trials: New York City’s teacher bonus program; the Triple P (Positive Parenting Program) System in South Carolina; and Hawaii’s Opportunity Probation with Enforcement (HOPE) Program.

In 2011, President Obama signed into the law the America COMPETES Reauthorization Act, which granted all federal agencies broad authority to conduct prize competitions to spur innovation, solve tough problems, and advance their core missions. To date, nearly 300 prize competitions have been implemented by 45 agencies through the website Challenge.gov. One of the leaders in conducing prize competitions has been NASA.

To learn more, we’re joined by Jenn Gustetic (@jenngustetic) from NASA. She’s the Prizes and Challenges Program Executive within the office of the Chief Technologist. Our discussion covers a recent example of how NASA has used prize competitions to solve an important challenge for the agency; the benefits of using prize competitions; the types of prizes used; and an overview of NASA’s Center of Excellence for Collaborative Innovation.

The City of Louisville is entering year three of its data-driven performance improvement initiatives under Mayor Greg Fischer, such as LouieStat that launched in January 2012. These efforts have gotten national attention. The mayor, for example, was recently recently named one of Governing Magazine’s Public Officials of the Year for his focus on improving the performance of city government. As the magazine noted, “At the heart of his performance efforts is a focus on data.”

We get an update on these initiatives from the city’s Chief of Performance Improvement, Theresa Reno-Weber (@RenoWeber). The interview discusses:

• LouieStat at year three: Louisville’s performance management system for metro government, called LouieStat, now includes 19 departments as well as one cross-functional, issue based “stat” program called VAPStat, which focuses on reducing vacant and abandoned properties.
• Problem-solving teams: For important issues that arise through the LouieStat process that require focused attention from issue experts, Louisville uses cross-functional teams created to address particular challenges. An example is the team created to improve a specific aspect of Emergency Management Services (EMS): reducing ambulance turnout times, meaning the time from patient drop off at the hospital to being ready for the next run. The team’s work, in just a four months, resulted in efficient gains worth $1.5 million, equivalent to two extra ambulances in service in the city.

For more background on Louisville’s performance initiatives, see Theresa Reno-Weber’s previous interview on the blog from 2013.

Can text messages be a useful and effective way for public agencies to communicate with citizens? If so, what elements of message design, including personalization (using people’s names), are most effective? And more broadly, how can public leaders at the local, state or federal levels use rapid and low experiments to pilot potential operational improvements to their programs or agencies and rigorously test if they work?

We examine these questions with Donald Green, Professor of Political Science at Columbia University and a leading expert in the design and implementation of field experiments. He is the co-author of a recent article in the Journal of Policy Analysis and Management about a large randomized trial conducted in England in 2012 to test the impact of different text messages on repayment rates for people who had delinquent fines owed to the government. The experiment was coordinated by the UK Behavioral Insights Team and an academic advisory panel, including Professor Green. 

Web extra: Donald Green discusses the potential to strengthen partnerships between academic experts and government agencies looking to design and test ways to improve their operations. [click here]

How can public leaders encourage greater use of program evaluation to learn what works and to improve outcomes for citizens? One important part of the answer, noted Paul Decker in his 2013 Presidential Address at the APPAM public policy conference, is to address the frequent fear associated with program evaluation. That fear stems from framing evaluation as simply a tool to justify continued or discontinued funding for a program. This “rigid framing of the role of program evaluation,” he argues, “poses a set of false choices that I believe ultimately undermines the use and creation of evidence.”

To learn more, we’re join by Paul Decker who recently concluded his year-long term as President of APPAM — the Association for Public Policy Analysis and Management. Since 2007, he has also been the President and Chief Executive Officer of Mathematica Policy Research. He is a nationally recognized expert in the design and implementation of evaluations of education and workforce development programs.

How can public officials move beyond guesses and hunches to more data-driven decision-making? One approach borrows from the health field, where randomized drug trials are a standard way to test the efficacy of potential pharmaceutical treatments. Leading companies also use randomized experiments — making operational changes to see if they work better — to improve their products and services. This same approach can be used in public policy, with individuals randomly assigned to a program group and a control group, in order to rigorously test what works and improve program performance. The approach is known as randomized controlled trials (RCTs) or impact evaluations using an experimental design.

To learn more, we’re joined by Rachel Glennerster, the Executive Director of the Jameel Poverty Action Lab or J-PAL based at MIT. She’s also the co-author, with Kudzai Takavarasha, of the new book Running Randomized Evaluations. It is a how-to guide for conducing valid randomized impact evaluations of social programs in developing countries. Our interview focuses on broader insights that are applicable to policymakers and public managers in the United States.

Web extra: Rachel Glennerster talks about some of the ethical issues involved in using randomized controlled trials in public policy. [click here]

How can a social service agency, whether at the state or local level, create an organizational culture focused on results? How can it create ongoing, meaningful conversations among agency leaders and staff that drive meaningful improvements? To explore these issues, we’re joined by Reggie Bicha (@reggiebicha), the Executive Director of the Colorado Department of Human Services under Governor John Hickenlooper and the Former Secretary of the Wisconsin Department of Children and Families under Governor Jim Doyle.

Our interview focuses on the performance leadership strategy that Reggie Bicha developed and launched in 2009 in Wisconsin called KidStat. As Secretary, he used the KidStat process to create alignment within the agency, helping ensure that “the right staff with the right resources delivering the right programs at the right time to the right people that’s going to help our state achieve what it is that I’ve been assigned by the legislator and governor to achieve.” Today, as head of the Department of Human Services in Colorado, Reggie Bicha continues to use a “stat” initiative to focus his agency on results, called C-Stat.

Web extra:  In our web extra, Nikki Hatch, who was formerly in charge of the KidStat process at the Wisconsin Department of Children and Families and today continues to work with Reggie Bicha as the Deputy Executive Director of Operations at the Colorado Department of Human Services, provides an example of how KidStat led to a specific improvement in performance. [click here]