What to Do When You Face the Hardest Question in Business

Have you ever bet enough money on something that it made you nervous?

I used to read a lot about finance, and at some point I came across an observation about something called convergence trading that stuck with me. A convergence trade is what you do when you expect the prices of two securities to eventually become equal; you basically bet that the higher-priced one goes down in price and the lower-priced one goes up. Sounds reasonable, but of course if the prices move further apart rather than converging you find yourself “quickly staring into the abyss,” as one author put it.

If you trust your original analysis, then you double down because the profit opportunity is now even larger. But doubling down when you’re wrong is suicide. Markets can stay irrational longer than you can stay solvent.

It’s hard to beat the stakes and drama of high finance, but over time I realized the convergence trade dilemma is just one version of what I believe is the hardest question in business.

When do you give up on an idea?

In my opinion, this is the hardest question there is in business:

Does it not work, or is it not working yet?

When you believe deeply in an idea – a product, a strategy, a trade, whatever – and the outcome doesn’t go as planned, you’ve ultimately got two options. Either admit defeat and cut your losses, or trust that it the plan will work and keep going. Few truly novel ideas just take off without the need for iteration, which means most spend at least some time in the purgatory of iteration.

When do you kill a product that isn’t selling? The more invested in it you are, the more agonizing the decision likely is. If you choose to “fail fast,” (which somehow escaped my business tropes that need to die column) you’ll never know if you gave up too early. Thinking about this recently, my mind drifted back to two of my favorite TV shows, both widely considered to be among the best, if not the best ever created in their respective genres: Arrested Development and The Wire.

Arrested Development was (sadly) cancelled after three seasons, while The Wire was allowed to run for its full five-season arc. Yet the ratings for both two shows bring us back to The Question:

Data source: arresteddevelopment.wikia.com

Cancelling Arrested Development after three seasons was justifiable for Fox based on the numbers – they had other much higher-performing shows at time. Then again, the fact that the fan base was large and rabid enough for Netflix to buy a fourth season ten years after the show was cancelled suggests they may have given up too soon. Meanwhile, The Wire ran to its planned conclusion without once hitting Arrested Development’s average viewership. Aside from critical acclaim (and, critically, DVD sales), it helped pave the way for dense, sprawling epics like Game of Thrones which are massive draws for HBO today.

When you’re faced with The Question, look for a new angle

Assuming you understand what your key success metrics are, there are two things you can do when the data shows your idea isn’t working like you think it should and you’re on the fence:

  1.  Find new metrics
  2. Find the missing context

Finding new metrics

Whether its sales, sign-ups, shares, downloads, or whatever else, if your key metrics don’t show the success you expect, it’s your job to figure out if that’s the whole story or just part of it. The ideal metrics to prove it may not exist, so you may have to create it yourself. Don’t be intimidated if that isn’t your thing. You can be data savvy without being a data scientist.

Start from first principles rather than the data. What will it take for your product to be successful? For instance, who are the early adopters you’re targeting? What would those people be saying about it? Where would you see residual effects of your products’ success? If those things were starting to happen, what would you expect to see? The art lies in connecting what you know in the abstract about your product with what you can observe in the data. Do that to the absolute best of your ability, and when you’re tapped out, get input from other people too. Then do the second thing.

Find the missing context

Is the data concealing important truths, good or bad? Let’s say your new app was download 10,000 times in its first week. If a big chunk of those will be vocal advocates and tell their friends, that could be a home run in the making. If your new users don’t have anything in common and there’s less to build on, that’s a worse outcome. The same numbers can mean different things.

This is something I have to do constantly with UserMuse. If I see a hundred new sign-ups come in, I have no idea if that’s meaningful or not. How many of these people are the type of users we need? Are they product managers looking for help with market research? Are they really interested in being an expert-for-hire and making money? Will they tell their friends, or are they the kind of people who sign up for everything? I need more than the headline data can give me. I have to email them one by one and ask what’s up. At this stage, there’s no other way to know.

I’m a data person, but like anything else data has its limits. There are things you can only learn by watching people and talking to them. So if you’re trying to figure out what’s going on in your store locations, for example, sometimes the best thing to do is close the spreadsheet and just go walk around some store locations. Go to a subway station and ask twenty random people what they think about your product. Cold email people you find on twitter or LinkedIn if they have relevant experience you can tap. Try to put a face on the data you’re seeing.

Whenever you face it, answering The Question is your job.