Home Page

If The Metric Is Valued, Someone Is Probably Trying To Game The System

February 11, 2019

by:

Joe Patti

Okay, so I promise I am not seeking out articles that discuss the problems with depending on quantitative metrics to determine effectiveness and value. They just keep falling into my lap. This one is via Dan Pink and is kinda fun to read thanks to some animations.

The piece in The Hustle has us follow the “career” of Otis has he moves from being a cashier to sales to online advertising to programming to surgery in order to illustrate how the use of quotas and efficiency metrics permeates every industry and every profession to incentivize gaming the system in order to generate the best appearance.

But Otis came to learn that metrics weren’t inherently bad — his bosses had just failed to grasp two important economic principles:

Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure,” and

Campbell’s Law: The more a metric is used, the more likely it is to “corrupt the process it is intended to monitor.”

He realized that when his performance was measured with a specific metric, he optimized everything to hit it, regardless of the consequences that arose. As a visiting professor at the London School of Economics told him, improper targets could:

Encourage “gaming” the system (e.g., bagging free groceries)

Incentivize the wrong aspects of work (e.g., writing trivial code)

Erode morale (e.g., writing clickbait)

Harm customers (e.g., turning away critical surgery patients)

And so, Otis decided to start his own company — a company where metrics would serve their true purpose: To motivate and align. Efficiency, Otis finally realized, isn’t just output; it is the value of what is produced.

If you think about the measures being applied to non-profit arts and cultural organizations like overhead ratio, economic impact, test scores, etc and pay attention to what organizations are doing in order to meet those metrics, you will probably start to see behaviors that conform to those listed above.

It could manifest as massaging numbers in financials and research; chasing funding that doesn’t align with mission and strains capacity; superficial efforts that check desired boxes; pursuit of a narrow segment of community rather than a focus on broader inclusion. I am sure readers can think of many examples from their own experiences.

Audience Relations, Building Public Will For Arts And Culture, State of the Arts, Terms and Ideas, Working In The Arts

No Comments

Cons Of Starting An Endowment

February 6, 2019

by:

Joe Patti

Recently there have been some conversations around my organization and town in general about whether it is worthwhile to try to bolster an existing endowment. People have mentioned that there has been a trend away from establishing endowments in recent years. I started wondering what the thinking behind that was and what the alternatives might be.

It just so happens Non Profit Quarterly reprinted a piece that talks about the pros and cons of establishing and endowment. On the con side is the issue is the idea that you are locking up money the organization could use now and disbursing it in the future when the same dollars don’t buy as much. Thirty years ago it may have seemed really attractive to learn that the organization would be receiving $25,000 a year from the endowment. In 1989 that could cover the salary for a position, but that money doesn’t go as far today. (Though there are plenty of places offering $25,000 as a salary.)

Current needs is the one that at least one of your board members will bring up, and is very possibly the reason why your board will vote not to have an endowment. “Why should we put a million dollars in a bank account when we can use that to serve a million more lunches?” Or buy a hundred thousand more books. Or facilitate a thousand more adoptions. Or renovate the façade of the theater. Many nonprofits are in dire need of more money, and most can at least think of an immediate way to use more….Some people go so far as to say it’s not ethical to lock money in the bank when there are so many necessary ways to spend it now. Before you know it, you have bad press and declining donations—and you wish you’d never thought of raising an endowment.

You may look jealously upon organizations receiving large payouts from their endowments, especially universities, but by and large these groups have a staff that is actively managing an endowment. A staff is required to grow an endowment to the size it yields enough to support your activities. But you need to have started with a large enough endowment to support a staff in the first place.

Finance, Fundraising

No Comments

What Is The Value Of Fire? How Do You Know?

February 5, 2019

by:

Joe Patti

Friend o’ the blog, Carter Gillies recently had a piece appear in the Arts Professional UK noting some of the problems with focusing on the instrumental value of the arts.

One of the issues he raises is the danger in making general claims about the value of the arts based on individual examples. One thing he cites that has been noted in other conversations on this topic is that if you tout the benefit of the arts to solve problems, you run the risk of something else coming along that does a better job and can be adopted as a replacement for the arts solution.

However, he points out that this also applies to employing problematic examples to make general statements about the lack of worth of arts and culture,

In fact, scepticism about the arts often does make exactly this type of argument: doubting their value in general, because there are obvious examples of offensive artistic work. They take these instances as being representative of the arts as a whole, when clearly they are not. And if we are combating such scepticism merely with the idea that some art actually does benefit society and individuals, then we have made the same mistake. The general case is not made or defeated with individual examples.

He also warns that an instrumental view of arts and culture can easily lead to the parsing of which forms of expression in particular are more effective at solving a particular ill. What is best at improving test scores? Does the same thing work for economic stimulus? (my emphasis)

Let’s think about what would follow if the point of art is its instrumentality. If it turns out that painting rainbows and unicorns is the most beneficial artistic practice, then we should start emptying museums right now. We have all the justification required to shed collections of Rembrandts, Picassos, and more.

My point is that the arts are valuable far and above their instrumental benefits. They weren’t invented to improve health and wellbeing outcomes. That they do is a happy coincidence. The arts aim at many things, and hardly ever directly at a particular cause. That is far too narrow a scope for understanding what the arts are, and why they matter.

As I have said many times, just because you can measure an effect doesn’t mean that measurement reflects the actual value of something. If there were more hot dogs and beer sold at the Super Bowl this week than the previous year, does that mean it was a better football game? Whether it is true or not is only a happy coincidence as Carter says, but it has no bearing on why people play or enjoy football to begin with.

Kids don’t organize games in their backyard or try out for local teams in the hopes of increasing hot dog sales in their community. Sure they had a winning season and exciting games before sold out crowds, but most people insisted on bringing potato salad from home instead of buying at the field, so sports are bad for the community by that measure. You may laugh because it seems ludicrous to use the sales of picnic food as a measure of success, but it is easy to get confused when presented with a measurement that is very important in some instances that isn’t necessarily relevant in others.

Which is the worse forest fire? The one that totally burns 50000 acres where no one lives or 50 acres with 15 houses valued at $2 million each? Which is more likely to cause people to denounce the value of fire in our lives? There are so many factors that contribute to forest fires and the discussion of management and prevention is complicated and nuanced. Not only can’t you use a few examples to make general statements about forest fires, the use of fire is so integral and entwined with our lives and who we are that you can’t use forest fires as a measure of the value of fire.

Advocacy, State of the Arts, Working In The Arts

No Comments

Collecting More Data Isn’t Necessary Better

February 4, 2019

by:

Joe Patti

Seth Godin offers a very relatable example of why more data isn’t always better by emphasizing the need for vigilance when setting an alarm clock in a hotel room. If you set the alarm for 7 am before going to bed at 10 pm but don’t notice that the clock currently reads 10 am, you aren’t going to be woken by the alarm clock the next morning. (I am sure we have all done this at least once.) He suggests the am/pm setting is an extraneous bit of data serving as an impediment to the clock fulfilling its purpose.

This is a very simple illustration of a point I bring up often on the blog — just because you can measure something doesn’t mean the data is useful for your goals and, in fact, may be an obstacle to understanding the relevant data. Just because you can measure the economic impact of the arts doesn’t mean economic impact is a valid measure of the value of arts and culture.

This concept also has relevance in terms of the regular practice of surveying audiences/attendees. Just because you can ask a question doesn’t mean you should or that what you learn will be useful.

As Godin writes,

The metaphor is pretty clear: more data isn’t always better. In fact, in many cases, it’s a costly distraction or even a chance to get the important stuff wrong.

Here are the three principles:

First, don’t collect data unless it has a non-zero chance of changing your actions.

Second, before you seek to collect data, consider the costs of processing that data.

Third, acknowledge that data collected isn’t always accurate, and consider the costs of acting on data that’s incorrect.

All this being said, my staff usually starts out surveys asking a question for which the answers will be useless as data points, but for which the goal is to establish a connection and willingness to respond in the survey taker. Basically, we figure people are more apt to answer 4-5 questions if the first one is a fun question about themselves. So for example, if we are doing Man of La Mancha, the question might be, “what is the impossible dream you dream?”

There are times when it is okay to collect data when it won’t shape your decision making and there might be a cost to collecting and processing it if doing so advances goals in other areas.

Audience Relations, Working In The Arts

No Comments