Kyle Bowen piqued my attention today in the Museums as Progress newsletter where he discussed Goodhart’s Law.
“When a measure becomes a target, it ceases to be a good measure.
Long time readers know that subscribe to the idea that just because you can measure something, it doesn’t mean the result is meaningful. Bowen illustrates this by pointing out that having goals to increase participation among people who meet certain demographic characteristics doesn’t advance your understanding of why they are there, how you can make them feel welcome, and make it easy to decide to return.
For a lot of organizations the answers to those questions are central to their mission and vision.
Goodhart’s Law reminds us that mirroring demographic ratios in museum content is not an end in itself because ratios have little relationship to a museum’s ability to fulfill a role in a city or region.
What counts isn’t who people are on paper, but what progress they’re trying to make in their lives. Demographics might tell you something about who is in your space, but they reveal nothing about why they came or what they hoped to achieve.
Bowen admits that it is possible for progress in their lives can fall victim to becoming a meaningless target, it is is more difficult to do. What everyone needs to achieve their goal is differs from person to person even if they have the same goal. Thus it is tough to focus on providing a standard solution to everyone. And because what everyone feels they need is specific to them, organizations have to engage in more direct and active listening to provide the outcomes community members seek. (emphasis original)
Supporting a community goal like “helping parents cultivate their child’s curiosity” requires understanding the diverse approaches people take in pursuing that goal and the alternatives people in a particular place might turn to as they seek to achieve their goal. You can’t reduce it to a single number. Second, progress metrics require ongoing listening rather than predetermined solutions. When you focus on supporting goals, you have to constantly validate whether your approaches are working, creating a natural correction mechanism.
