Which experts should you trust?

Taking advice from experts

Do you believe experts who are confident in their predictions? Philip Tetlock says this might be a mistake.

Take weather forecasters. You won’t be surprised to learn that the UK weather forecasters do not have a very good reputation for the accuracy of their forecasts. But wait; there are forecasters who do. Apparently those in the US have a better track record.

It turns out that there are two reasons for this. The first is that the US forecasters tend to give their predictions (which is what a forecast is) in terms of percentages; “60% chance of rain tomorrow” Where as ours predict with more certainty; “It will rain tomorrow.”

The other reason he says, for the better track record, is that the US forecasters take more effort to get feedback and learn from it. Feedback is essential for learning. You cannot improve if you don’t get feedback.

It turns out that from his research there are two types of expert. Those who are not even as good at predicting as tossing a coin would be, and those who are better than chance.  This does not just apply to weather forecasters.

Unfortunately Philip Tetlock discovered that there is an inverse correlation between ‘fame’ and ‘accuracy’.

This turns out to be because the media likes those who seem to be certain, not those who say they’ don’t know’ sometimes. Which in turn means that we, the views, are forcing the issue.

We like certainty

The irony is that the more certain experts are, the less reliable they are but the more we believe them.

It seems we just don’t like uncertainty.

Two lessons

Be more ready to trust experts who are not certain and are willing to say they don’t know.
Make sure you get feedback so that you can improve any predictions you make.

A simple experiment to try

One of the key skills in time management is being able to estimate how long tasks will take accurately. This is something that many people are very bad at and for some reason, seem to make no effort to improve.

Here’s a very easy way to improve your estimation skills in anything. It will often take less than a week to see really drastic improvements.

Choose your of task

It could be doing your emails, phoning a customer or updating your expenses.

Make your estimate

Write down how long you think it will take (I usually put this straight into my schedule).

Check your estimate against your prediction (feedback)

This is pretty easy. If you estimated 10 minutes and it took 20 you are 100% out.
Make your next estimate, and correct it based on your previous estimate
In my example you need to double the time you estimate for that task next time.

Very soon you will be able to estimate times quickly and accurately.

Lastly

Do not dismiss people who are cautious with their predictions or who say they don’t know. They probably know more than the people who tell you they are certain.

An easy way to stop learning

Did you know that there is no learning without feedback? I heard Prof. Marian Diamond speak on this topic many years ago.  She’s an expert in the brain and is still working at 85.

She is one of the world’s leading neuroscientists. I was lucky enough to meet her. Till I heard her talking about feedback, I had no idea how important it was. I thought it was dry and boring. Just something you had to do in a performance review or appraisal and perhaps once or twice a year.

Now I know better. The important thing to realise is that by not giving or getting enough feedback, you are not learning as much as you could be. You are making your life more difficult than it has to be. You are not moving forward and improving.

No learning

Have you seen people (or companies) who make the same mistake again and again? We all have.

One of the key reasons for this is just not checking information you already have about the results of your actions. You often don’t need to get feedback from others; you can get the information yourself. And that’s all feedback is: information.

It’s information that helps you to make a decision on what to do next time.

A small example of feedback working

I worked with Anne, a finance manager. She had a team member who was unreliable. He often did not complete tasks when they were required. Many of her requests were sent by email.

We printed them all out and she put them into two piles. The ones he had complete on time and the ones that had not been completed on time.

There was a clear different in the way she had asked him to complete the task. When she had given a specific time or date, the tasks were done promptly. Where she had said they were ‘urgent’ or ‘really important’ (with not date) they had not been completed on time.

So her feedback was clear – to get tasks completed on time, Anne needed to give a specific date.


Find out how to get feedback and make sure others get it.