Professional regulators and law-enforcement officials certainly deal with customers, and so the normal range of customer-service metrics has some relevance for them. But their principal business is to deliver protections to society against various classes of harm, and towards that end they deliver and, where necessary, enforce obligations, influence behaviors, and manage compliance. The nature of “success” to be pursued and assessed is therefore quite different in the “protective” role of government than in service-provision roles.
Many public agencies select three or four metrics they think are particularly important, and designate them as “Key Performance Indicators,” drawing to them an inordinate amount of internal and external attention. Sometimes this focus can have perverse effects and bias operations in a way that is not quite as valuable as executives intended and may even distort operations in a way that is contrary to the public interest.
For regulators, much attention is typically paid to functional outputs (such as inspection or audit rates), to the rates at which agents discover or detect non-compliance, and to process metrics such as timeliness and accuracy of decision making. Connecting these measures to each other, and to valuable social outcomes, remains a complex puzzle.
This seminar provides an opportunity for attendees to assess their own organisation’s traditions in performance measurement and reporting, and—where necessary—to identify opportunities to broaden the range of indicators and analyses in a way that will enable their agency to present a more rounded and meaningful picture of their contributions to society.
Who should attend?This course is designed for mid to upper level regulatory and enforcement practitioners.
Choose your level of depth:• Day 1 is designed to cover the core issues
• Day 2 is designed to dive deeper into some specific topics
• Participants may sign up either for Day 1 alone, or for Day 1 & Day 2
• Those who previously participated in the October 2020 version of this workshop (the content of which closely matches the Day 1 agenda outlined below), are welcome to come back and join us for discussion of the “deeper dive” issues on Day 2 only. *The option to sign up for Day 2 only is not available to others.
Key Learning OutcomesDay 1 - Core Concepts
• Understanding distinct types of regulatory work (functional, process-based, risk-based, & crisis-response), and the classes of performance metrics relevant to each
• The notion of an “information rich environment,” and making the distinction between management information & performance metrics
• The multiple purposes of performance measurement
• Where “customer-service” metrics fit within regulatory operations, & where they do not
• Classic ambiguities in assessing regulatory performance (e.g. When would finding more violations be good news, or bad? How should we interpret increased volumes of complaints?)
• What difference it makes to the performance story when regulators turn their attention to “risk-based” strategies
Day 2 - Deeper Dive
• What it takes to “prove causality”. The meaning of “evidence-based” policy and practice, and how these ideas apply to regulatory & enforcement agencies
• The role of program evaluation. Where it fits naturally within regulatory operations. Other forms of analysis and pattern recognition that count
• The distinctive challenges of performance measurement when tackling invisible risks, catastrophic risks, and risks involving conscious opponents (adversaries)
• The implications of different regulatory structures on performance monitoring and evaluation
• Seminar participants will also have the chance to submit in advance your own performance measurement puzzles. While we cannot promise to address them all, we will incorporate some, of broad general interest, into the agenda.
Some of the testimonials from the 2020 seminar:“Professor Sparrow generously shares his wisdom and brilliant insight with such ease and good humour. His session on Defining Success & Measuring Regulatory Performance is thoughtprovoking, practical and should be mandatory for all medium to senior-level regulators.”
- Environmental Protection Authority NSW, October 2020
“Outstanding session with application potential in range of settings - highly recommend.”
- Worksafe Victoria, October 2020
“Professor Sparrow’s session on defining and measuring regulatory performance was enlightening. He clearly articulated what a regulator should and shouldn’t consider as a metric and what the pitfalls are in trying to present management activities as performance metrics”.
- NSW Department of Primary Industries, October 2020
This seminar is:
- Popular: Our highest attended & rated seminar of 2020
- Highly Rated: With 250+ participants in 2020, Professor Sparrow received a customer satisfaction rating of 98%
- Tangible Takeaways: 87% of the audience were inspired to make a change in their organisation after attending this seminar
- Great for Groups: Group bookings are highly popular, with the opportunity to work through the concepts as a team
Malcolm K Sparrow is a leading international expert in regulatory and enforcement strategy, security and risk control. Malcolm is the Professor of the Practice of Public Management at Harvard’s John F Kennedy School of Government and he is Faculty Chair of the school’s executive program “Strategic Management of Regulatory and Enforcement Agencies.”
Professor Sparrow served 10 years with the British Police Service, rising to the rank of Detective Chief Inspector. He has conducted internal affairs investigations, commanded a tactical firearms unit, and has extensive experience with criminal investigation. His research interests include regulatory and enforcement strategy, fraud control, corruption control, and operational risk management.
Professor Sparrow holds an MA in mathematics from Cambridge University, an MPA from the Kennedy School, and a PhD in Applied Mathematics from Kent University in Canterbury.
What People Are Saying
“Professor Sparrow generously shares his wisdom and brilliant insight with such ease and good humour. His session on Defining Success & Measuring Regulatory Performance is thoughtprovoking, practical and should be mandatory for all medium to senior-level regulators.”Environmental Protection Authority NSW
“Outstanding session with application potential in range of settings – highly recommend.”Worksafe Victoria
“Professor Sparrow’s session on defining and measuring regulatory performance was enlightening. He clearly articulated what a regulator should and shouldn’t consider as a metric and what the pitfalls are in trying to present management activities as performance metrics”.NSW Department of Primary Industries