Tamler Sommers and Michael Twa
“When a measure becomes a target, it ceases to be a good measure.” Goodhart’s Law
Recently, a scandal rocked higher education, as multiple universities' efforts to
increase their rejection rates in order to move up in the US News college rankings
board came to light. Whose fault was this? Was it US News for using rejection rates
as a metric for “good” education? Was it the fault of college administrators trying
to game the system? Over the last two decades, there has been an increasing obsession
with metrics in domains as diverse as education (e.g. standardized testing), criminal
justice, social science research, medicine, and management. What are the cost, benefits,
and hidden assumptions behind this new emphasis? Do metrics offer more accurate, less
biased information or merely a veneer of objectivity that crowds out human judgment
and discretion?
Have you ever looked at a dataset and thought, “how did they come up with these numbers?”
Have you ever wondered how a business can claim to be ranked #1 in customer satisfaction?
How is satisfaction measured, anyhow? At this table we will discuss all of these problems
and many more.