Which experts should you trust?

Taking advice from experts

Do you believe experts who are confident in their predictions? Philip Tetlock says this might be a mistake.

Take weather forecasters. You won’t be surprised to learn that the UK weather forecasters do not have a very good reputation for the accuracy of their forecasts. But wait; there are forecasters who do. Apparently those in the US have a better track record.

It turns out that there are two reasons for this. The first is that the US forecasters tend to give their predictions (which is what a forecast is) in terms of percentages; “60% chance of rain tomorrow” Where as ours predict with more certainty; “It will rain tomorrow.”

The other reason he says, for the better track record, is that the US forecasters take more effort to get feedback and learn from it. Feedback is essential for learning. You cannot improve if you don’t get feedback.

It turns out that from his research there are two types of expert. Those who are not even as good at predicting as tossing a coin would be, and those who are better than chance.  This does not just apply to weather forecasters.

Unfortunately Philip Tetlock discovered that there is an inverse correlation between ‘fame’ and ‘accuracy’.

This turns out to be because the media likes those who seem to be certain, not those who say they’ don’t know’ sometimes. Which in turn means that we, the views, are forcing the issue.

We like certainty

The irony is that the more certain experts are, the less reliable they are but the more we believe them.

It seems we just don’t like uncertainty.

Two lessons

Be more ready to trust experts who are not certain and are willing to say they don’t know.
Make sure you get feedback so that you can improve any predictions you make.

A simple experiment to try

One of the key skills in time management is being able to estimate how long tasks will take accurately. This is something that many people are very bad at and for some reason, seem to make no effort to improve.

Here’s a very easy way to improve your estimation skills in anything. It will often take less than a week to see really drastic improvements.

Choose your of task

It could be doing your emails, phoning a customer or updating your expenses.

Make your estimate

Write down how long you think it will take (I usually put this straight into my schedule).

Check your estimate against your prediction (feedback)

This is pretty easy. If you estimated 10 minutes and it took 20 you are 100% out.
Make your next estimate, and correct it based on your previous estimate
In my example you need to double the time you estimate for that task next time.

Very soon you will be able to estimate times quickly and accurately.

Lastly

Do not dismiss people who are cautious with their predictions or who say they don’t know. They probably know more than the people who tell you they are certain.