How can public agencies use a test, learn and adapt approach to improve their results? One way is to learn from leading private sector firms that use rapid experimentation to learn what works and improve their operations.

To learn more we are joined by Jim Manzi, the founder and chairman of Applied Predictive Technologies, a business analytics firm. His 2012 book Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics and Society argues for the usefulness of experimental methods—in other words, randomized controlled trials (RCTs)—for addressing important policy issues, from improving education outcomes to increasing economic growth to reducing crime.

In a review of Uncontrolled in the New York Times, columnist David Brooks writes, “Manzi wants to infuse government with a culture of experimentation.” Brooks also notes: “What you really need to achieve sustained learning, Manzi argues, is controlled experiments. Try something out. Compare the results against a control group. Build up an information feedback loop. This is how businesses learn. By 2000, the credit card company Capital One was running 60,000 randomized tests a year — trying out different innovations and strategies. Google ran about 12,000 randomized experiments in 2009 alone.”

Steve Aos is the Director of the Washington State Institute for Public Policy (WSIPP). The Institute’s mission is to carry out practical, non-partisan research—at legislative direction—on issues of importance to Washington State. Areas of focus have included education, criminal justice, welfare, children and adult services, health and more. The outcome of WSIPP’s work includes cost-benefit analyses of various policy options so that legislatures can make more informed decisions about cost-effective policies.

Although not discussed in the interview, a related initiative to note that builds on WSIPP’s work in Washington State: The Pew Charitable Trusts and the MacArthur Foundation are currently working with 14 states to develop and strengthen their cost-benefit analysis capabilities, as part of the Results First Initiative.

Web extra: Steve Aos explains how trust between WSIPP and the legislature, build over time, is an important aspect of the Institute’s ability to produce analysis that is valued and used. [click here]

Mike Powell is the Chief Innovation Officer for the State of Maryland under Governor Martin O’Malley. In that role, he helps oversee the Governor’s Delivery Unit.

For public leaders in the United States, the Governor’s Delivery Unit is a useful example of setting specific, ambitious strategic goals, of being transparent with citizens about progress, and of creating a process internally — with close leadership involvement — to “own” those goals and drive progress towards them.

In particular, the Delivery Unit is designed to help achieve 15 strategic goals that are important to citizens and that cut across traditional agency silos. Its website tracks progress, including graphs, background information, and flags that show whether each goal has been delivered, is on track or progressing, or has had insufficient progress so far.

Mike previously worked for IBM as a consultant and was a CityStat analyst in Baltimore earlier in his career.

As background, it’s useful to note another O’Malley Administration initiative mentioned in the interview: StateStat. Every few weeks or months, depending on the agency, each agency leadership team comes to a StateStat meeting in the Governor’s Office to review the agency’s performance data and to discuss, with the Governor’s staff, ways to improve outcomes. The Governor’s Delivery Unit, in contrast, is closely related (and managed by StateStat staff) but is focused on cross-cutting goals that span agencies and that have been identified by the Governor as key benchmarks for state progress.

Web extra: Mike Powell discusses his role as Chief Innovator Officer for Maryland, answering a question that, as he has noted, he gets frequently: “What does a Chief Innovation Officer do?” [click here]

Amelia Showalter served as Director of Digital Analytics on Barack Obama’s 2012 presidential campaign, leading a team that designed and implemented hundreds of experiments to improve the performance of all types of digital outreach.  Today, she is a consultant who helps organizations achieve better results through testing.

In the interview, we draw on her innovative experience in politics to consider how government — in nonpartisan ways — can learn from the same techniques of testing and experimentation in order to improve program and agency performance. In other words, how can public managers build a culture of rapid organizational testing and improvement?

Web extra: Amelia Showalter talks about the “email derby” in which she and her staff tried to pick the most and least effective versions of fundraising emails and how the derby results underscored the usefulness of randomized testing. [click here]

Matt Leighninger is the Executive Director of the Deliberative Democracy Consortium and the author of the recent report for public managers, “Using Online Tools to Engage – and be Engaged by –The Public” published by the IBM Center for the Business of Government. The motivation behind the report: While many federal and state agencies are striving to engage with citizens using online tools and e-government, many public managers find themselves unfamiliar with what tactics and tools work best under different scenarios.

What does it take to make telework a successful tool for public agencies and workers? To explore that question, we’re joined by Scott Overmyer. He’s the author of the IBM Center for The Business of Government report Implementing Telework: Lessons Learned from Four Federal Agencies. The report offers practical implementation advice to agency leaders and front-line managers.

As background, the Telework Enhancement Act of 2010 expands telework opportunities to over one million federal workers. Today federal agencies are working to implement that law, while state and local officials are also findings ways to promote telework. Telework — working outside of the office — has the potential to save billions of taxpayer dollars along with other benefits.

Almost half of all pregnancies in the United States are unintended and the women and children involved in these pregnancies are disproportionately likely to experience a range of negative outcomes. For insights into cost-effective policy strategies to reduce unintended pregnancy, my guest is Adam Thomas, a professor at Georgetown University and the former Research Director for the Brookings Institution’s Center on Children and Families. His recent article in the Journal of Policy Analysis and Management — as well as a Brookings policy brief— focus on three evidenced-based strategies to prevent unintended pregnancy and the results from fiscal impact simulations of those strategies. His findings have implications for both federal and state policymakers.

Jeff Tryens is the Deputy Director for Performance Management in the Mayor’s Office in New York, under Michael Bloomberg. He is responsible for reporting overall city government performance and initiating cross-agency performance improvement initiatives. Prior to that role, Jeff  was the Executive Director of the Oregon Progress Board from 1995 to 2005. In that role, he helped oversee Oregon’s strategic vision, known as Oregon Shines as well as its 90 indicators of social, economic and environmental health known as Oregon Benchmarks. Those efforts helped Oregon became a leader in performance reporting.

Web extra: Jeff Tryens talks about his current work in New York City, including what is unique about New York City’s performance reporting, including the Mayor’s Management Report (MMR) [click here].

The U.S. Department of Housing and Urban Development (HUD) has 9,000 employees and a $46 billion budget. How does the agency focus its numerous staff and diverse programs on critical goals? It uses HUDStat, launched in 2010.

Each quarter, key staff related to each goal from various departments within HUD come to a HUDStat meeting, led by HUD Secretary Shaun Donovan, to review performance data and discuss ways to boost results. By having detailed data at hand, the participants can understand what’s working and what’s not. By having key staff all in one room, questions can get answered quickly. Each HUDStat meeting produces specific action items that are then reviewed in the next meeting.

HUD currently has seven priority goals, including ones related to homelessness, foreclosure, and energy efficiency. That means there are seven HUDStat meetings per quarter, each focusing on a different goal.

In some ways, HUDStat is similar to other “PerformanceStat” initiatives such as CitiStat in Baltimore. In other ways, it is relatively unique, including the fact that meetings focus on cross-agency goals (not specific departments) and that the agency Secretary leads the meetings rather than one of his deputies.

To learn more about HUDStat, we’re joined by Lisa Danzig, the Director of Strategic Planning and Management for HUD.

A note to listeners: The audio is choppy in a few places due to the connection.   

Web extras: Lisa Danzig discusses…

• her advice to federal, state, or local agency leaders who are interested in adopting a HUDStat-like initiative [click here].
• a benefit of tabulating and sharing data through the HUDStat process: positive “peer pressure” to increase results [click here].
• the motivation behind the launch of HUDStat [click here].
• HUD’s current priority goals (recall that each HUDStat meeting focuses on a different goal, with every goal covered once per quarter) [click here].
• staff work involved in preparing for HUDStat meetings and some upcoming advances in the model [click here].

A growing trend among cities and states in the U.S. is outcome budgeting. Baltimore began using outcome budgeting in Fiscal Year 2011 under the leadership of Mayor Stephanie Rawlings-Blake and her budget director — and our guest on the Gov Innovator blog — Andrew Kleine. The results include more dollars to high-value programs, the reduction or elimination of low-value programs, and new thinking and innovation in the delivery of services to citizens.

Web extra: Andrew Kleine gives advice to jurisdictions that may be considering implementing outcome budgeting, including the use of consultants and also staffing issues [click here].