Probability of Pandemic --- When (If Ever) Will Experts Give Us a Number?
As a statistician (and a probabilist) I would greatly appreciate the work and courage of any influenza experts who would take on the challenge of putting a number on the likelihood of an H5N1 1918-level pandemic.
Many public health officials have made statements in the style of "Not a matter of IF but of WHEN." These are surely true, and perhaps they are the most responsible statements one can make with the current state of knowledge.
Still, do these statements really help? That the sun will eventually evolve to a red star and vaporize the earth is an absolute fact of stellar evolution --- but so what? The sun's evolution offers no cause for concern on any human time scale.
For personal planning, what I and others like me would greatly wish is an estimate for probability of a pandemic in 2006, 2007, 2008, etc. This is, I believe, a rational desire, but is it a realistic desire?
Why Essentially All Experts Will Pass
They will decline from offering a probability estimate for good reasons, the biggest being that there is no established methodology for making such estimates. They could try speaking from the gut and just put a number on what they feel. Unfortunately, this is not likely to add much value.
Influenza experts have so far shied away from quantitative statements about the likelihood of a 1918-level pandemic, but after the usual not-if-but-when statement many experts have added something personal about the preparations they have taken for their families. This is a genuine measure of seriousness --- as are the expenditures that have been undertaken by governments. Still, these measures fall far short of a probability estimate.
Are Probability Estimates "Taboo"?
One question that has been asked is if probability estimates are taboo in the public health community. It certainly is not the custom to provide such estimates, and there is not even any evidence that they are encouraged.
This may be for good reasons. First, most consumers do not know what to do with a probability estimate, and, second, most experts do not know how to reframe their insight into an appropriate probability estimate.
How About Trying?
While I would encourage influenza experts to try to put their personal beliefs into the language of subjective probability, I would not think less of them if they simply said they could not. Moreover, though I would appreciate their efforts to form probability estimates, I would not bet the ranch that they would get it right.
People are great at probability estimates when they have the experience of repeated trials (Poker, Bridge ... even baseball), but with one-time events (or few-times events) humanity has no competence to estimate probabilities. Even retrospective estimates may baffle us. How close did Khrushchev come to launching his missiles during the Cuba crisis? The question has been discussed for forty years and we still have no honest answers.
What Would Be Ideal?
I believe that we already possess a rich enough scientific base for the probability estimation problem to be moved beyond the simple subjective probability estimates of individuals.
We know the mechanisms that matter --- shift, reassortment, etc., and there is substantial understanding of the kinds of changes that would be needed for current H5N1 strains to achieve an H2H efficient mutation. Mutation rates can be estimated --- or at least guessed by analogy from historical data on other viruses. Many --- but not all --- of the puzzle pieces are in place.
If a group of virologist and probability modelers gave priority to the task, we could obtain honest probability estimates. This would be useful now and useful in the future.
Reference:Tversky, Amos and Kahneman, Daniel,"The Framing of Decisions and the Psychology of Choice." Science v. 211 no. 4481, 1981.
Many public health officials have made statements in the style of "Not a matter of IF but of WHEN." These are surely true, and perhaps they are the most responsible statements one can make with the current state of knowledge.
Still, do these statements really help? That the sun will eventually evolve to a red star and vaporize the earth is an absolute fact of stellar evolution --- but so what? The sun's evolution offers no cause for concern on any human time scale.
For personal planning, what I and others like me would greatly wish is an estimate for probability of a pandemic in 2006, 2007, 2008, etc. This is, I believe, a rational desire, but is it a realistic desire?
Why Essentially All Experts Will Pass
They will decline from offering a probability estimate for good reasons, the biggest being that there is no established methodology for making such estimates. They could try speaking from the gut and just put a number on what they feel. Unfortunately, this is not likely to add much value.
- Without training in probability (and a gambler's good sense) a virologist (or other) is unlikely to be very good at making probability predictions --- even about bland day-to-day events, much less life changing events.
- Studies initiated by Amos Tversky and Daniel Kahneman and continued by many others suggest that almost everyone tends to treat 20% events as "impossible" and 80% events as "certain". About the only people who do much better at understanding long odds are those who make regular bets, such as actuaries, options traders, and bookies.
- Without data (and something like repeated trials) the only bases for probability estimates are futures market prices and the intuitions of informed individuals. Unfortunately, as I mentioned in an earlier piece on TradeSports, the current futures markets are too thin to be used. This leaves us with the experts.
Influenza experts have so far shied away from quantitative statements about the likelihood of a 1918-level pandemic, but after the usual not-if-but-when statement many experts have added something personal about the preparations they have taken for their families. This is a genuine measure of seriousness --- as are the expenditures that have been undertaken by governments. Still, these measures fall far short of a probability estimate.
Are Probability Estimates "Taboo"?
One question that has been asked is if probability estimates are taboo in the public health community. It certainly is not the custom to provide such estimates, and there is not even any evidence that they are encouraged.
This may be for good reasons. First, most consumers do not know what to do with a probability estimate, and, second, most experts do not know how to reframe their insight into an appropriate probability estimate.
How About Trying?
While I would encourage influenza experts to try to put their personal beliefs into the language of subjective probability, I would not think less of them if they simply said they could not. Moreover, though I would appreciate their efforts to form probability estimates, I would not bet the ranch that they would get it right.
People are great at probability estimates when they have the experience of repeated trials (Poker, Bridge ... even baseball), but with one-time events (or few-times events) humanity has no competence to estimate probabilities. Even retrospective estimates may baffle us. How close did Khrushchev come to launching his missiles during the Cuba crisis? The question has been discussed for forty years and we still have no honest answers.
What Would Be Ideal?
I believe that we already possess a rich enough scientific base for the probability estimation problem to be moved beyond the simple subjective probability estimates of individuals.
We know the mechanisms that matter --- shift, reassortment, etc., and there is substantial understanding of the kinds of changes that would be needed for current H5N1 strains to achieve an H2H efficient mutation. Mutation rates can be estimated --- or at least guessed by analogy from historical data on other viruses. Many --- but not all --- of the puzzle pieces are in place.
If a group of virologist and probability modelers gave priority to the task, we could obtain honest probability estimates. This would be useful now and useful in the future.
Reference:Tversky, Amos and Kahneman, Daniel,"The Framing of Decisions and the Psychology of Choice." Science v. 211 no. 4481, 1981.
posted by J. Michael Steele at 12:39 PM
2 Comments:
I think that there are additional political reasons for a "taboo" on probability estimates. An expert might believe that the bird flu will happen, but can only give it a 10% chance of happening next year. That is scary to someone who understands probability and repeated events, but does that scare politicians enough to do something about it? Probably not.
So, the subsequent temptation is to increase the probabilities to the point where people will really begin paying attention, but that has the associated risk that you will be branded as an alarmist if things don't go as expected. Almost everybody focuses on the expectation of a stochastic process without taking into account the variance of that process.
The bottom line difficulty in my opinion is that few have the staying power-- financial (liquidity, etc), political (credibility, etc) and psychological (hedonic treadmill, etc)-- to sustain the drawdown associated with repeated favorable bets on low probability events, even if one is an extremely intellectually capable individual. This shrinks the incentive for people of all disciplines to make those forecasts in the first place.
Nassim Taleb has said that his gains from '87 and the bubble bursting put him well in the black for decades. He has written some very interesting things on the topic of black swans and LPE's, all for free on his website.
Post a Comment
<< Home