Margen
Up to ten dollars on the table and one decision: tie them to your own forecasts or to a program's. That is what was put in front of 361 people in a lab at the Wharton business school in Philadelphia. The task was to guess what percentile each of a set of MBA students would end up in, from eight items in their application: GMAT score, essay, interview, years of work experience, previous salary. The program was an ordinary statistical model, built from those same data.
The program did not get it right. On average it was off by 22 percentiles —twenty-two places in every hundred—. The people were off by 26.6.
Four and a half percentiles of difference, measured on the forecasts that actually paid money. On that task the humans accumulated between 15% and 29% more error than the model.
And here is the part nobody expects. Some of the participants were shown the program working before choosing: fifteen forecasts, with its hits and its misses in plain view. Those were the ones who chose it least.
Confidence in the model was measured on a 1-to-5 scale. People who had not seen it work gave it 3.04. People who had seen it work gave it 2.49. Meanwhile, among those who saw a person err —and the people erred more— confidence in that person did not drop appreciably: 2.70 without seeing her and 2.47 having seen her, a difference the authors could not separate from noise.
And it is not just confidence in the abstract. Before watching it work, participants estimated that the program would land within five percentiles almost half the time: 46.5%. After watching it work, that same estimate dropped to 28.2%. The program was the same one, and the fifteen examples they had just seen were real.
Put the other way round: watching a program make a mistake disqualifies it, and watching a human make one does not.
The most uncomfortable detail is in the group that was shown both things, the program and the person, side by side. Across the five runs of the experiment, 610 of those 741 people saw with their own eyes the model beat the human. That group was, in every study, among those least likely to choose the model. They saw the whole comparison, they saw who won, and then they bet on the loser.
The authors called it algorithm aversion, and it was published by Berkeley Dietvorst, Joseph Simmons and Cade Massey in 2015. In every condition of every study, the participant would have earned more money by choosing the model. In the first one, 1.78 dollars on average against 1.38.
The explanation they offer is not that people think machines are stupid. Quite the opposite: before seeing it run, the model is assumed to be almost perfect. The problem is that "almost". A program is required not to fail, and at the first failure its credit is withdrawn entirely; a person is granted failure because failure was already priced in. The human is forgiven in advance.
Three years later the same authors published the other half of the finding, and this is the useful part. They tried letting people touch the model's output: see its forecast and move it. With that, the choice changed: they used it, they defended it and they were more satisfied. The astonishing bit is how much they were allowed to move it. Some, ten percentiles; others, five; others, two. It made no difference. The size of the margin changed nothing appreciable; what changed things was having a margin.
This is not about statistical models. It is about the quote that comes out of your template and that you adjust by eye because this client is different, about the list of symptoms the emergency doctor runs down, and about the points system by which places on a waiting list are handed out. All those rules have one thing in common: they are made out of your own past experience, and the moment you will throw them out is right after watching them fail once.
So the amount of margin you are going to allow yourself gets decided beforehand, not afterwards, and it gets written on the same sheet as the rule: how much I may move the number it gives me, and in what units. Ten per cent up or down, two weeks, two hundred euros.