If The Metric Is Valued, Someone Is Probably Trying To Game The System

Okay, so I promise I am not seeking out articles that discuss the problems with depending on quantitative metrics to determine effectiveness and value. They just keep falling into my lap. This one is via Dan Pink and is kinda fun to read thanks to some animations.

The piece in The Hustle has us follow the “career” of  Otis has he moves from being a cashier to sales to online advertising to programming to surgery in order to illustrate how the use of quotas and efficiency metrics permeates every industry and every profession to incentivize gaming the system in order to generate the best appearance.

But Otis came to learn that metrics weren’t inherently bad — his bosses had just failed to grasp two important economic principles:

  • Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure,” and
  • Campbell’s Law: The more a metric is used, the more likely it is to “corrupt the process it is intended to monitor.”

He realized that when his performance was measured with a specific metric, he optimized everything to hit it, regardless of the consequences that arose. As a visiting professor at the London School of Economics told him, improper targets could:

  • Encourage “gaming” the system (e.g., bagging free groceries)
  • Incentivize the wrong aspects of work (e.g., writing trivial code)
  • Erode morale (e.g., writing clickbait)
  • Harm customers (e.g., turning away critical surgery patients)

And so, Otis decided to start his own company — a company where metrics would serve their true purpose: To motivate and align. Efficiency, Otis finally realized, isn’t just output; it is the value of what is produced.

If you think about the measures being applied to non-profit arts and cultural organizations like overhead ratio, economic impact, test scores, etc and pay attention to what organizations are doing in order to meet those metrics, you will probably start to see behaviors that conform to those listed above.

It could manifest as massaging numbers in financials and research; chasing funding that doesn’t align with mission and strains capacity; superficial efforts that check desired boxes; pursuit of a narrow segment of community rather than a focus on broader inclusion. I am sure readers can think of many examples from their own experiences.

Still More Engaging Perspectives on Museum Collections

In the last six months it seems like I have been coming across a lot of stories about how museums tours are presenting alternative contexts for collections. Back in May I was writing about Museum Hack which is providing tours through a somewhat pop culture lens.

Since then I have come across a number of stories about efforts in Philadelphia Museums to provide tours from a number of different perspectives, including those who have lived and played around the ruins of ancient civilizations.

Last week I saw a story in the NY Times about a pilot program University of Cambridge has launched that to provide an LGBTQ+ perspective on their collections. The university recruited Dan Vo who had already established himself as a figure in alternative museum tours to help them develop their series.

His Polar Museum tour highlighted artfully carved whale teeth known as scrimshaw — a way of occupying male whalers so that they didn’t have sex with each other, Mr. Vo said — and items from indigenous communities that showed how fluid gender roles were in some Arctic populations.

Tours like these are important for the future of museums, Mr. Vo said in an interview later. “It makes them relevant,” he said, “and people want to see themselves reflected in collections.”

The article quotes Alistair Brown, policy officer at the Museums Association who says museums

“are looking at radical ways of reappraising their collections,” he said in a telephone interview. “They’re either inviting critical and diverse voices into the museum, or at least welcoming their presence if uninvited.”

Some of the tours aren’t as welcome as others. The NY Times also highlighted Uncomfortable Art tours given by Alice Procter which highlight the imperialism and colonialism underlying museum acquisitions. She has received death threats because promotional images on her website label Queen Elizabeth I as a slaver and Queen Victoria as a thief. The British Museum created a tour series of their own in response to Procter’s which highlights the specific provenance of objects in their collection.

Examples like these (including the ones I cited in earlier posts) can provide a real sense of the potential inherent in museum collections and the type of things people are curious to know.  It also highlights the type of details arts organizations should know about their offerings whether it is museum objects or works being performed.

It has only just started to occur to me that this is a result of the development of Professional-Amateurs predicted 15 years ago and has become something that can both challenge and threaten arts organizations and greatly enhance the work they do.

Trees Come With Unexpected Baggage

In my post yesterday I referenced the difficulty non-profit arts organizations have with conducting outreach activities that have relevance to communities. I and others have also frequently written about the problems with the way arts organizations approach relations with underserved communities, especially communities of color.

The honest truth is, a lot of non-profit organizations find the work they are doing has poor resonance with the communities they hope to serve. I was reading a piece on CityLab today about an organization that is trying to plant trees in Detroit. You would think this is a pretty non-controversial endeavor, but many neighborhoods in Detroit had a narrative of distrust in which trees figured prominently.

But as I read the article, I felt like so many phrases and terminology were exactly the same ones that crop up in discussions about how arts organizations need to frame their approach and relationships with underserved communities.

For example,

Elliot Payne, described experiences where green groups “presumed to know what’s best” for communities of color without including them in the decision-making and planning processes.

“I think a lot of the times it stems from the approach of oh we just go out and offer tree plantings or engaging in an outdoor activity, and if we just reach out to them they will come,” Payne told Taylor.

Cut out the references to tree planting and outdoor activity and it immediately sounds like a conversation at an arts conference without even needing to insert arts terminology.

Then there was this passage:

However, from reading excerpts of Carmichael’s interviews with TGD staff members, it’s clear some of the tree planters thought they were doing these communities an environmental-justice solid. After all, who would turn down a free tree on their property, given all of the health and economic benefits that service affords? Perhaps these people just don’t get it. As one staff member told Carmichael in the study:

You’re dealing with a generation that has not been used to having trees, the people who remember the elms are getting older and older. Now we’ve got generations of people that have grown up without trees on their street, they don’t even know what they’re missing.

How many times have you been part of a conversation where those advocating for the value of the arts talk exactly along these lines? – People don’t understand the value of the arts and the benefits they afford. The younger generation isn’t used to attending/participating in arts experiences. They have grown up without arts educational classes or opportunities to attend performances, they don’t even know what they’re missing.

What was really interesting to read was how residents of neighborhoods and the city were operating from two different narratives about trees. A researcher was surprised to learn that nearly a quarter of the 7500 residents the tree planting organization approached rejected the trees. When the researcher spoke to residents, they told her about how the city cut down the elm trees that used to line the streets after the 1967 race rebellion so that it was easier for police to conduct helicopter surveillance. The city, on the other hand, said they cut down the trees and sprayed them with DDT from helicopters in order to stem the spread of Dutch Elm disease which threatened during that time.

It was this conflicting narrative that motivated residents to reject the trees. They were already well aware of the benefits of trees in providing shade, improving home values, filtering air pollution, etc., it was just that they didn’t trust the motivations of the city.

This made me wonder if people were more aware of the benefits of the arts than we believe and there are narratives that inform a sense of distrust. Ideas about what the arts are and who they are for may comprise a large part of that narrative.

There was also a far more practical consideration fueling the rejection. People felt someone else was deciding what should be planted and where without having any conversations with the people who would have to live with the trees —and rake the leaves and branches that fell. The city doesn’t have the resources to trim the trees or remove dead ones that threaten the fall so the residents would bear the consequences.

What I could really empathize with was that The Greening of Detroit, the organization planting the trees, probably felt like they were doing a lot to have conversations and involve the community in a discussion about the tree planting.  In retrospect, there were missteps in their approach and they didn’t dedicate enough staff resources to outreach. However, they held community meetings and placed door hangers, both of which discussed their plans and their commitment to maintain the trees for three years following the planting.

Unfortunately, none of these things made the right connection with the residents but I could see a lot of arts organizations in similar circumstances feeling that making the investment to take those steps was doing a good job by residents.

It seems like the really, really retail, one-on-one interactions that were part of the researcher’s follow up was what was needed to make residents satisfied they were being heard.

One Detroit resident whom Carmichael interviewed for her study told her: “You know what, I really appreciate you today because that shows that someone is listening and someone is trying to find out what’s really going on in our thoughts, the way we feel, and I just appreciate you guys. And maybe next time they can do a survey and ask us, if they would like to have us have the trees.”

California Symphony–They Speak Your Language

I was excited to see Aubrey Bergauer posted a follow up to her original 2016 Orchestra X post regarding how the California Symphony was acting on the feedback it has received about the concert planning and attending experience. I have written about some of Aubrey’s work since then, but I was eager to see a cumulative reflection.

Unfortunately, her post came in the middle of the holiday production crunch so I only got around to reading it this week.

A couple of really interesting things that caught my attention in this latest post. First was the counter-intuitive value in leaving past events posted on the website. I always want to get the clutter of old information off my website so it is easy for potential attendees to find the information they want. While this is probably an important practice generally, for the California Symphony, leaving that information available helped bolster their credibility. She writes,

1) As the season progressed, this list got awkwardly short, especially for an orchestra like the California Symphony that doesn’t perform as frequently as our bigger-budget peers. Participants told us they couldn’t believe we didn’t perform more often, and it looked even worse when only a few concerts were on that list. 2) As they were trying to “get a sense of what we’re about,” as they said, they couldn’t really tell based on only a handful of upcoming shows

Another thing is that they started running digital ads in both English and Spanish. The Spanish ads have a link to a Spanish language landing page.

That pilot test did lead to a measurable increase in Latinx households, and so we decided to put some money behind developing the new site in both languages. Now, when we run ads in Spanish, we can link to landing pages in the same language, another step in making this important segment in our community feel invited and welcome here, as well as give them the information they need to join us.

This was not new information to me because Aubrey has been reporting her success attracting a broader audience segment on Twitter for a few weeks now.

While she didn’t report on the outcomes of the changes, her discussion of how they adjusted some of the website sections to be outwardly focused rather than inwardly focused gave me something to think about. For example, instead of “Education” as a navigation header they are using “Off Stage” with subheaders focused on kids, adults and artists. They also changed “Support Us” to the more outwardly oriented “Your Support.”

A lot of the work they did was in the area of providing background information both in their program book and website. Their program notes are more about the background of the artists and music than the technical details of the music. They have song clips and information drawn from Wikipedia available online for those who want to know more. They changed their writing style to short bullet points rather than paragraphs.

Aubrey provides the rationale behind these changes based both in research and user feedback so it is definitely worth while to read this recent post.