For-Credit Winter Graduate Course
Improving Programs
through Evaluation-Based Performance
Monitoring and Management
9-20 January, 2023
Milan and Rome, Italy

The University of Maryland School of Public Policy will offer an in-person, for-credit (and a parallel, non-credit) international course on evaluation-based, performance monitoring and management. Classroom instruction will be provided by university instructors and guest faculty—complemented by consultations with officials of the Italian Government, international organizations, and NGOs.
Classes will be held at the Milan campus of the Università Cattolica del Sacro Cuore as well as other locations in Milan and Rome.
This course is part of the University of Maryland’s new “Certificate of Professional Studies in Program Planning, Monitoring, and Evaluation.” As such, it is designed as a three-credit, stand-alone course for doctoral, master’s level, and advanced undergraduate students studying social policy-related subjects such as economics, education, international development, political science/governance, public administration, public health, public policy, social welfare, sociology, and urban planning.
Mid-career professionals may also take the course individually or as part of the four-course Certificate. There is also a non-credit option for mid-career professionals who are interested in learning the material and attending classes, but who do not want to complete the graded coursework. 

Course Summary

Even the best designed program can falter from poor implementation or management. Consequently, governments, multinational organizations, NGOs, and funders seek to improve program operations—using the tools of program evaluation to monitor program activities, outcomes, and costs. The most important tools they use include counterfactual identification, performance targets, and comparative effectiveness evaluations.
This, in turn, creates a demand for staff able to apply and interpret these tools of evaluation-based program monitoring. The course seeks to advance the skills and careers of participants by helping them understand and use these tools in real-world settings. Students will learn how to:

• Formally assess the need to establish, modify, or expand a program (whether public or private);
• Use logic models to identify and operationalize key elements of performance indicators;
• Collect and analyze data performance data for program monitoring;
• Use counterfactuals, performance targets, and comparative effectiveness evaluations to gauge performance;
• Estimate program costs (including total costs, cost-efficiency, cost-effectiveness, and cost-benefit analysis); and
• Apply the resulting data to improve program performance though management techniques and program adjustments and restructuring.

In addition to course instruction, the class will also visit senior officials of relevant Italian government offices, international organizations, and NGOs in Milan and Rome. The Rome meetings will include the Bank of Italy; Italian National Institute for the Evaluation of the Education and Training System (INVALSI); and Italian Senate, Committee on Labour and Social Security.
A preliminary syllabus will be available on the course web page.
The course will culminate in the preparation of a set of performance measures for either an existing program familiar to the student, a proposed program, or a program that is suggested by the instructors. Students will prepare a logic model for the program, use the logic model to identify and define, operationalize performance measures, describe how data will be collected for the performance measures on an ongoing basis, and, for outcome measures, identify the method for identifying counterfactuals that will be used for each of the measures. Students will then evaluate each of the performance measures using criteria discussed in the course.
Final papers will be due at the end of March 2023.
Classes will be in English.

Faculty 

The core faculty will be:

• Douglas J. Besharov, Professor, University of Maryland, College Park (Program Director);
• Douglas M. Call, Lecturer, University of Maryland, College Park; and
• Gian Paolo Barbetta, Professor, Catholic University of the Sacred Heart, Milan, Italy

The faculty will be augmented by a number of guest faculty who are luminaries in the fields of program evaluation and performance measurement, tentatively including:
• Burt Barnow, Professor, George Washington University;
• Neil Gilbert, Professor, University of California, Berkeley;
• Naomi Goldstein, Deputy Assistant Secretary for Planning, Research, and Evaluation; Administration for Children and Families, US Department of Health and Human Services (ret.); and
• Anu Rangarajan, Senior Vice President and Director of Climate Change, Mathematica Policy Research

Course Schedule and Facilities

Prior to the class meetings in Milan, a preliminary meeting will be held (online) to discuss the course syllabus and other arrangements. In Milan, the course will meet daily (tentatively 9:00-1:30) on weekdays from 9 January through 14 January, 2023.
The course schedule in Rome will vary, depending on the availability of the institutions the class will visit.
As mentioned above, final papers will be due at the end of March, after students leave Rome.

Eligibility
The course is open to all potential students, including individuals enrolled in accredited graduate and undergraduate programs as well as professionals who are
seeking additional training and education. Enrollment in a University of Maryland program is NOT required.
Students who are currently enrolled in a graduate program must be in good academic and judicial standing and must not have any registration blocks. The minimum GPA for this program is 3.0 (unless otherwise approved by the program director).
Professional and non-credit students must have, at the minimum, a bachelor’s degree or equivalent experience.

Course Credit
Students will receive three credits from the University of Maryland upon successful completion of the course. Non-UMD students should confirm the transferability of credits with their home academic departments.
Participants in the non-credit version of the course will receive a “citation of participation.”

Course Cost

Standard program fee: 3,500 USD. The “early bird” fee (for students who apply before November 15, 2022) is 3,000 USD. (Both are fully refundable up to two weeks from commencement of the course.)
Enrollment Fee: 75 USD (for non-UMD students only; non-refundable).
The program fee includes registration and administration, tuition, course related activities, and teaching services. Students are responsible for transportation (including to and from Milan, and to and from Rome) as well as lodging, meals, and other incidental expenses.
A non-credit version of the course will also be offered (at a cost of 2,000 USD).

Application Deadline
Applications will be accepted through January 2, 2023. To receive the reduced “early bird” rate, applications must be submitted by November 15, 2022.
Because space is limited, early application is encouraged. Students will be accepted on a rolling basis.

Application
Prospective students should contact Michael Horlick (mhorlick@umd.edu) of the University of Maryland School of Public Policy to apply for the course.

Accommodations
Students will be responsible for arranging their own accommodations in Milan and Rome.

Additional Information
Further information regarding the course format, travel, and other course logistics will soon be available on the course web page.

Contact
For any specific questions regarding the course, please contact Michael Horlick (mhorlick@umd.edu) of the University of Maryland.