[Cross-posted to both the Extropians list, and the "everything" list at http://www.eskimo.com/~weidai/everything.html. To help concentrate the discussion, please direct followups to the latter mailing list.]
On Wed, Jun 12, 2002 at 06:32:04PM -0700, Hal Finney wrote: > For example, if I am offered a choice between running a very pleasant > experience twice identically, or running the very pleasant experience > once and a slightly less pleasant experience once, which should I pick? > On the one hand, the average pleasantness is higher in the first > alternative. On the other hand, I have two different experiences in > the second alternative. I don't know which is better, so I don't have > an initial subjective preference between them. Earlier (at http://www.lucifer.com/exi-lists/extropians/2612.html) I argued that preference between the two choices is subjective (i.e. depends on your utility function). I now realize this implies that the self-sampling assumption (or SSA, the idea that you should reason as if you were a random sample from the set of all observers, see http://www.anthropic-principle.com/index.html for more details) cannot be applied universally, because it implies that only choosing the two identical experiences is rational. Here's a demonstration of this. Suppose you've agreed to participate in the following experiment. First you're copied. The original will observe while the copy (named A1) is told the following. A1 will be copied into A2, B1 and B2. All four will be run on seperate and identical computers. A1 and A2 will be shown a number equal to the millionth bit in the binary expansion of PI. B1 and B2 will both be shown a number equal to 1 minus that bit. All four will be asked to guess the millionth bit of PI. (Assume you have no idea what the millionth bit is.) If A1 guesses correctly, it will experience a very pleasant experience (call this experience E1). Same applies for B1 and B2, each of whom will also have E1 if he guesses correctly. If A2 guesses correctly however, he will experience a slightly less pleasant experience E2. If anyone guesses incorrectly, he's halted immediately. In any event all four copies are halted at the end of the experiment. (The setup can be changed so that the four runs are done sequentially instead of in parallel. I don't think that affects my argument at all.) Now put yourself in the position of A1 before he's been further copied, trying to devise a strategy for guessing the millionth bit of PI. Let's call that bit X and the number you'll be shown Y, and consider the two strategies A) guess Y, and B) guess 1-Y. It should be obvious at this point that if you prefer to have two identical very pleasant experiences you'll select strategy B, and if you prefer to have one very pleasant experience and one slightly less pleasant experience you'll select strategy A. However according to the SSA only strategy B is rational. Here's how I would analyze the situation given the SSA. After being shown Y, there's 1/4 probability that I'm A1, 1/4 probability that I'm A2, 1/4 probability that I'm B1, and 1/4 probability that I'm B2. So if I guess Y, there's 1/4 probability that I cause a copy of me to experience E1 and 1/4 probability that I cause a copy of me to experience E2, therefore my expected utility is U(A) = 1/4*U(E1) + 1/4*U(E2). If I guess 1-Y instead, there's 1/4 probability that I cause a copy of me to experience E1 and another 1/4 probability that I cause a copy of me to experience E1, so my expected utility is U(B) = 1/2*U(E1). Since U(E1) > U(E2), U(B) > U(A). Here's my proposed non-SSA way of analyzing the situation. After being shown Y, I consider myself to be A1, A2, B1, and B2 "simultaneously". If I guess 1-Y, there's probability of 1 that I cause two copies of me to experience E1 (call this {E1,E1}). If I guess Y, there's probability of 1 that I cause one copy of me to experience E1 and one copy of me to experience E2 (call this {E1,E2}). Now which strategy I should choose depends on whether U({E1,E1}) > U({E1,E2}), which can be independent of whether U(E1) > U(E2). So my position is that rather than being a principle of correct reasoning, the status of the SSA should be reduced to that of an approximation useful when one's utility function is close to satisfying certain constraints (for example U({E1,E2})=U(E1)+U(E2) for all E1, E2). More general principle(s) need to be worked out that subsumes the SSA as a special case.
