Monday, November 7, 2011

A quick note on log odds

Probabilties are usually numbers that range from 0 to 1. However, the standard way of representing probabilities is not always optimal. However, there is another mapping of probability that goes from negative infinity to infinity. This system called "log odds" has a number of advantages. In the standard log odds approach, one maps the probability of an event x to the quantity log(x/(1-x)). Brian Lee and Jacob Sanders wrote a good summary (pdf) of this system which discusses its advantage and disadvantages. As they observe, use of log odds allows one to immediately see how something like the change in probability from 51% to 52% isn't that big whereas the change from 98% to 99% is a much larger change in the sense that the chance of the event not happening has now halved. Log odds helps makes this sort of intuition immediatelty obvious from the numbers. Brian and Jacob discuss the advantages and disadvantages of log odds in detail, and show how it is particularly useful for doing Bayesian updates. I strongly recommend reading their piece.

5 comments:

MKR said...

I have two reactions to the proposal of Lee and Sanders.

(1) One might, rightly or wrongly, take it to show that there is nothing natural or necessary about the use of the interval [0, 1] (I hope that I am using the correct notation; I am not a mathematician) to quantify probabilities. The origins of the mathematical theory of probability lie in the attempt to compute odds in gambling. It seems natural that the classical and frequentist interpretations of the theory would arise out of such a context, and the use of ratios, or percentages, or numbers in the interval [0, 1] seems the most natural expression of probabilities conceived according to those interpretations (viz., as the ratio of favorable outcomes to total "equally possible" outcomes, or as the relative frequency of favorable outcomes in an infinite series of outcomes, respectively). But if we think of probabilities in epistemic terms, as degrees of confidence, expectation, or belief--which is what I take to be meant by the term "Bayesian" in these discussions--then, custom and historical influences aside, the linear system (I hope that I am using the correct term) does not seem more natural than the logarithmic one. One can even argue plausibly, as Lee and Sanders do, that the logarithmic one is more natural.

(2) My other thought is that there is less to the proposal than meets the eye. One question that arose in my mind after I read the paper was, what becomes of the axioms of the theory of probability on this proposal? It would seem that the Kolmogorov axioms cannot be used, as they are expressed in the linear system. So (I thought) we must introduce a new set of axioms. But then by what right do we presume to be still doing the mathematics of probability? Then it occurred to me that the log-odds system is not a new way of doing probability theory at all but simply a new way of crunching the numbers. The log odds are derived from and converted back into the linear representation of probabilities based on the Kolmogorov axioms (or whatever set of axioms). So the log-odds sysemt is analogous to what Microsoft Windows was in relation to MS-DOS when it was first introduced, namely a mere "shell" or user interface. (Back in the 1990s, I asked a friend of mind who worked, none too contentedly, at Microsoft, "Is Windows an operating system?" He replied sardonically, "Microsoft would like you to believe that it is.")

--Miles

Joshua said...

I agree that the proposal isn't that massive a change. In that regard, we can for any reasonably nice range do probability in that range. (We could for example translate things over to any interval we wanted say [3/2,44]. The question is whether a given setting is more or less useful for thinking. Very often in math and science equivalent formulations of the same ideas can emphasize different aspects or make working with different aspects more or less useful.

I agree with the analysis about the Bayesian approach, although I think it is worth noting that Bayesian updating is essentially mathematical. Using it as an epistemological framework is a philosophical decision.

I don't think it would be that hard to translate standard axioms like Kolmogorov into this sort of framework although I haven't gone through the effort to do so.

Sniffnoy said...

Wait a minute... Brian Lee, Jacob Sanders... I didn't even look at the names before! Now I have to wonder if that's the same Brian Lee and Jacob Sanders of BCA...

Sniffnoy said...

It occurs to me that emphasizing the log odds point of view may be counterproductive when dealing with people new to the idea that epistemology should be quantitative/probabilistic, and who are uncomfortable with less than absolute certainty. It's correct that the difference between 100% and 99% is in some sense infinitely large, while the difference between 51% and 50% isn't; but that's not what you want to be emphasizing in such a case! I mean, you could explicitly say that for decision theory (expected utility) purposes the difference is just 1% either way, while the infinite difference is purely an epistemological matter, but I suspect that's not going to be the most helpful introduction.

Anonymous said...
This comment has been removed by a blog administrator.