Opinion

“THE DRUNKARD’S WALK”

Harvard rejected a record 91 percent of applicants this year. This doesn’t mean those turned down were unworthy – perhaps just unlucky. Randomness rules our lives more than we’d care to admit. Can we really say with certainty that an accepted application was better than one that was tossed aside, or that a $90 bottle of wine is superior to a $10 bottle, or the winner of the “world championship series” in our favorite sport is really the best team? As a student of randomness, I have learned that the answer to all three questions is a resounding “no!”

My son Alexei provided a case in point: he told me two of his friends once turned in identical essays for a school assignment. Not only did the overworked teacher not notice, she gave one of his friends a 90 (an A), and the other a 79 (a C), sending one boy home to a pat on the back, and the other to a grilling about too much TV – both by parents convinced that the grade reflected something real.

Was the grade disparity a fluke? I looked into it. My favorite study involved researchers who collected 120 term papers and treated them with a degree of scrutiny your own child’s work will never receive: each paper was scored independently by eight faculty members. The resulting grades, on a scale from A to F, sometimes differed by two or more grades. On average, they differed by nearly one grade. Similar results were found by many others, including when the University of Texas analyzed the scores on its college-entrance essays. If a publisher could have rejected the manuscript for “Diary of a Young Girl” by Anne Frank, calling it, “very dull” and “a dreary record of typical family bickering,” it might be too much to ask a teacher to distinguish so finely between class essays. Yet on such judgment calls often rest the futures of our children.

*

The psychologist William James suggested that some wine experts could be so attuned as to judge whether a sample of Madeira came from the top or bottom of a bottle. But most scientific studies show that people even have a hard time judging whether it came from a bottle or a box. Setting, mood and peer pressure contribute at least as much to wine ratings as the subtle scents of green pepper, wild strawberries or “freshly tanned leather.”

For example, in one recent study, in a group of volunteers rated a bottle labeled $90 in a lineup significantly higher than another bottle labeled with a $10 price tag even though the sneaky researchers had filled both bottles with the same wine. We tend to err on the side of praise rather than criticism lest we be ranked lower by our peers. The catch is, this test was conducted while the subjects’ brains were being imaged, and the image showed that the pleasure center of the brain really did light up more when they drank the “more expensive” bottle. Another group of researchers found that experts presented with three wines, two of which were identical, could identify the odd wine only about two-thirds of the time, and when asked to rank wines based on components such as tannins, sweetness and fruitiness, they disagreed significantly.

Given these studies, can the numerical wine grading system that millions of people rely on have any validity? There hasn’t been a large scale study of this issue, but a few years back “The Penguin Good Australian Wine Guide” and “On Wine’s Australian Wine Annual” provided an Alexei test: they both reviewed the 1999 vintage of the Mitchelton Blackwood Park Riesling. “The Penguin Guide” named the wine Penguin Best Wine of the Year. “Wine Annual” rated it the worst vintage if the decade.

*

Though the examples I’ve given relate to the subjective measurement of quality, it is important to understand that variability is an issue even when measurements reflect concrete and objective characteristics. For example, in sports, suppose history demonstrates that one team can be expected to beat another in 55 percent of their games. Mathematics tells us that in a best-of-seven series the inferior team will win about four times out of ten. The odds of the better team winning improve if we lengthen the series. Still, in this example, for the weaker team to be crowned champion 5 percent or less of the time (the standard for “statistical insignificance”) a championship series would have to be 269 games long. The same mathematics applies in business. If a CEO has, say, a 60 percent probability of success each year, the chances that in a given 5-year period the CEO’s performance will reflect that underlying rate are only about 1 in 3.

I guess that’s what makes life exciting, but it’s also something to think about before you place your bets, pick your wine, or ban your children from watching TV because their essay grade wasn’t up to snuff.

Leonard Mlodinow’s new book, “The Drunkard’s Walk: How Randomness Rules Our Lives” (Pantheon) is out this month.

Excerpts from “The Drunkard’s Walk”:

Which is greater: the number of six-letter English words having “n” as their fifth letter, or the number of six-letter English words ending in “ing”? Most people choose the group of words ending in “ing.” Why? Because they’re easier to think of than generic six-letter words. But the group of six-letter words having “n” as their fifth letter includes all six-letter words ending in “ing.” Psychologists call this the availability bias, because we give unwarranted importance to memories that are most vivid and most available for retrieval.

Suppose the state of California offered its citizens the following game: Of all those who pay the dollar or two to enter, most will receive nothing, one person will receive a fortune, and one person will be put to death in a violent manner. Would anyone enroll? They do – it’s called the state lottery. Applying statistics from the National Highway Traffic Safety Administrations, you find a reasonable estimate of driving fatalities is about one per game.

Are streaks in sports real? Not according to a 1985 study by researcher Amos Tversky. He investigated reams of basketball statistics and found that better players have a higher chance of scoring, but their chances of a successful basket after scoring a basket was the same as after a failure.