The Government Performance and Results Modernization Act of 2010 (GPRAMA) was intended to provide federal leaders with objective program effectiveness information and aid in federal agency decision-making. However, a Government Accountability Office (GAO) survey of federal managers, GAO-13-518, indicated a widespread absence of program evaluations, as well as a lack of progress in the use of available performance information.
In order to identify areas for improvement, the GAO recently conducted a study, Program Evaluation: Some Agencies Reported that Networking, Hiring, and Involving Program Staff Help Build Capacity. The study reviewed the capacity of federal agencies to conduct and use program evaluations.
Program evaluations are methodical analyses that review specific issues of program performance. While program measurement generally tracks progress against goals determined at the program outset, program evaluation“typically assesses the achievement of a program’s objectives and other aspects of performance in the context in which the program operates.”
One objective of the GAO study was to identify activities that are useful in building capacity to conduct and use program evaluations. The GAO study surveyed performance improvement officers at 24 agencies, which included DoD and civilian agencies. According to the study, the most useful activities for increasing capacity to conduct evaluations include:
- Hiring – Recruit personnel with research and analysis experience. Some agencies use hiring programs such as Presidential Management Fellows, Intergovernmental Personnel Act or the American Association for the Advancement of Science fellows program.
- Professional Networking – Encourage staff to participate in conferences or related interest groups. One example mentioned by those surveyed was the Association for Public Policy Analysis and Management Research Conference.
- Consulting experts – Consider consulting with experts for theoretical or technical support; identifying areas in which agency staff may need additional assistance.
- Training – Build staff capacity through specific skills and knowledge training. Personnel may benefit from classroom or online training in areas such as data and statistical analysis techniques, design of program evaluations, and converting evaluation results into agency recommendations.
- Accountability – Holding leaders accountable and conducting quarterly progress reviews were mentioned by several study respondents as beneficial for improving agency ability to conduct evaluations.
Once an agency is successfully conducting evaluations, it is essential to apply the acquired information to improve outcomes. The performance improvement officers surveyed identified quarterly progress reviews, engaging staff, and enforcing accountability for agency goals as activities which promote the use of evaluations in decision-making. In particular, including staff throughout the program evaluation process generates early buy-in of evaluation findings.
Developing capacity to conduct and use program evaluations can allow agencies to be better informed for future decisions. With some intentional planning, infrastructure activities such as hiring practices, networking, training, support services and accountability structure can be harnessed to improve outcomes and ultimately, mission objectives.