gts wrote:
Hi Ben,
On Extropy-chat, you and I and others were discussing the foundations
of probability theory, in particular the philosophical controversy
surrounding the so-called Principle of Indifference. Probability
theory is of course relevant to AGI because of its bearing on decision
theory (I assume that's why you invited me here. :)
As you know, the Principle of Indifference (PI) states that if no
reason exists to prefer any of n possibilities then each possibility
should be assigned a probability equal to 1/n. The PI is known also as
the Principle of Insufficient Reason, the name given it by classical
probabilists who followed after Laplace, who took it for granted as a
self-evident principle of logic. (It was John Maynard Keynes who later
renamed it the Principle of Indifference.)
I found a discussion of the Principle of Insufficient Reason in this
book about decision theory:
Choices: An Introduction to Decision Theory By Michael D. Resnik
http://books.google.com/books?vid=ISBN0816614407&id=4genrKNUkKcC&pg=RA2-PA35&lpg=RA2-PA35&ots=wE4Uxk7bqE&dq=principle+of+insufficient+reason&sig=PsMUy3fqcMgFha8Kyx2HLaC-EA8
This author criticizes the PI in two ways. His first is mainly
philosophical: if there is no reason for assigning one set of
probabilities rather than another, then there is no reason for
assuming the states are equiprobable either. This is pretty much the
same argument I was trying to make on ExI.
His second objection is one we had not discussed: though the PI seems
like a common-sense way to proceed under conditions of uncertainty,
invoking it can sometimes lead to disastrous consequences for the
decision-maker. I would add that while the PI might be useful in some
situations as a heuristic device in programming AGI, perhaps some
accounting should be made for the extra risk it entails.
-gts
You have a point, but perhaps not the one you think.
The principle of indifference states that you should consider the
probabilities of each case to be 1/n, it doesn't say anything about the
relative costs of acting as if each was of equal weight. Generally if a
situation is potentially more dangerous (not increased probability of
harm, but probability of increased harm) the reward for not avoiding it
will need to proportionately FAR greater.
This means that even though the probabilities of occurrence must be
deemed equal, one shouldn't consider them equally unless the cost of
each event occurring is also equal. As such, indifference may be a poor
name.
As to whether the assumption of equal probability is valid...it seems a
reasonable default position, but should be invested with very low
certainty. This means that one should try to avoid any plans based on
the presumption that equal probability is correct, but doesn't mean that
there is a better default position.
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
