In his book 'How To Measure Anything', author Douglas Hubbard writes about a simple test you can conduct to check if your confidence in a belief is real or imaginary.
It is called The Equivalent Bet Test. And this is how it works.
Suppose you're asked to give a 90% Confidence Interval (CI) for the year in which Newton published the universal laws of gravitation.
In simpler language, you're asked to give a range of years in which you think Newton published the work a range that makes you feel at least 90% confident of the actual year falling under your given range. If you were to bet on it, there would be a 90% chance of you winning the bet.
Now, time to win some money. Once you've made your guess, a friend comes and tells you that I will give you two ways in which you can win $10,000:
1. You can win $10,000 by betting on your answer, i.e., you win if the true year of publication falls within your given range of years. Otherwise, you win nothing.
2. Or, you can win by spinning this dial which is divided into two 'pie slices," one covering 10% of the dial, and the other covering 90%. If the dial lands on the 10% slice, you win nothing. If it lands on the 90% slice, you win $10,000.
Which option do you prefer more to win the $10,000?
If you honestly find yourself preferring option 2 and think it's an easier way to win those ten thousand dollars, then you must think spinning the dial has a higher chance of winning you $10,000 than option 1.
That suggests your stated 90% confidence in your answer isn't really 90%. Otherwise, why would you choose spinning the dial, which offers the same odds?
By preferring option 2, your brain is trying to tell you that your originally stated 90% CI is overconfident, and it probably is really or 70% or 60%.
If instead, you find yourself preferring option 1, then you must think there is more than a 90% chance your stated range of years contains the true value.
For contrast, imagine your friend had offered you to spin a dial on which the first pie slice covered 40% of the dial and you wouldn't win anything if the dial landed on this 40% pie.
Would you then readily pick option 1 — betting on your answer — versus option 2?
In general, what % of the dial should be covered by the first pie in order to conclusively make it a more guaranteed way to win the $10,000 than option 1? That will tell you how confident you are in your answer.
If you say that even if you covered 55% of the dial and it would still be a better bet, then you might not be very confident in your answer and it would be as good as a coin flip — which is 50% odds of winning.
To make a better estimate, adjust your answer (range of years) until option and option 2 with 10% of the dial covered seems equally good to you. That means that you're at least 90% confident that the actual year Newton published his laws lies under this range.
Why does this test work for real-world decision making?
Because it's easy to be confident when the stakes are low and there's not a lot to lose. But when the stakes are high and when there's real money on the table, that's when you to need to really think about how confident you are about a certain decision.
I've found it to be a good trick to force honesty in your decision-making. When ten thousand dollars is on the line, your brain automatically tells you the truth.
That is also why having skin in the game is important to make your opinions count. Because if you're wrong, it is you who will have to pay for them. $10,000 to be precise.
For a real business example of upping the stakes and having skin in the game, read this essay.
