[Cross-posted to both the Extropians list, and the "everything" list at 
http://www.eskimo.com/~weidai/everything.html. To help concentrate the 
discussion, please direct followups to the latter mailing list.]

On Wed, Jun 12, 2002 at 06:32:04PM -0700, Hal Finney wrote:
> For example, if I am offered a choice between running a very pleasant
> experience twice identically, or running the very pleasant experience
> once and a slightly less pleasant experience once, which should I pick?
> On the one hand, the average pleasantness is higher in the first
> alternative.  On the other hand, I have two different experiences in
> the second alternative.  I don't know which is better, so I don't have
> an initial subjective preference between them.

Earlier (at http://www.lucifer.com/exi-lists/extropians/2612.html) I 
argued that preference between the two choices is subjective (i.e. depends 
on your utility function). I now realize this implies that the 
self-sampling assumption (or SSA, the idea that you should reason as if 
you were a random sample from the set of all observers, see 
http://www.anthropic-principle.com/index.html for more details) cannot be 
applied universally, because it implies that only choosing the two 
identical experiences is rational.

Here's a demonstration of this. Suppose you've agreed to participate in 
the following experiment. First you're copied. The original will observe 
while the copy (named A1) is told the following. A1 will be copied into 
A2, B1 and B2. All four will be run on seperate and identical computers. 
A1 and A2 will be shown a number equal to the millionth bit in the binary 
expansion of PI. B1 and B2 will both be shown a number equal to 1 minus 
that bit. All four will be asked to guess the millionth bit of PI. (Assume 
you have no idea what the millionth bit is.) If A1 guesses correctly, it 
will experience a very pleasant experience (call this experience E1). Same 
applies for B1 and B2, each of whom will also have E1 if he guesses 
correctly. If A2 guesses correctly however, he will experience a slightly 
less pleasant experience E2. If anyone guesses incorrectly, he's halted 
immediately. In any event all four copies are halted at the end of the 
experiment. (The setup can be changed so that the four runs are done 
sequentially instead of in parallel. I don't think that affects my 
argument at all.)

Now put yourself in the position of A1 before he's been further copied, 
trying to devise a strategy for guessing the millionth bit of PI. Let's 
call that bit X and the number you'll be shown Y, and consider the two 
strategies A) guess Y, and B) guess 1-Y. It should be obvious at this 
point that if you prefer to have two identical very pleasant experiences 
you'll select strategy B, and if you prefer to have one very pleasant 
experience and one slightly less pleasant experience you'll select 
strategy A. However according to the SSA only strategy B is rational.

Here's how I would analyze the situation given the SSA. After being shown 
Y, there's 1/4 probability that I'm A1, 1/4 probability that I'm A2, 1/4 
probability that I'm B1, and 1/4 probability that I'm B2. So if I guess Y, 
there's 1/4 probability that I cause a copy of me to experience E1 and 1/4 
probability that I cause a copy of me to experience E2, therefore my 
expected utility is U(A) = 1/4*U(E1) + 1/4*U(E2). If I guess 1-Y instead, 
there's 1/4 probability that I cause a copy of me to experience E1 and 
another 1/4 probability that I cause a copy of me to experience E1, so my 
expected utility is U(B) = 1/2*U(E1). Since U(E1) > U(E2), U(B) > U(A).

Here's my proposed non-SSA way of analyzing the situation. After being 
shown Y, I consider myself to be A1, A2, B1, and B2 "simultaneously". If I 
guess 1-Y, there's probability of 1 that I cause two copies of me to 
experience E1 (call this {E1,E1}). If I guess Y, there's probability of 1 
that I cause one copy of me to experience E1 and one copy of me to 
experience E2 (call this {E1,E2}). Now which strategy I should choose 
depends on whether U({E1,E1}) > U({E1,E2}), which can be independent of 
whether U(E1) > U(E2).

So my position is that rather than being a principle of correct reasoning, 
the status of the SSA should be reduced to that of an approximation useful 
when one's utility function is close to satisfying certain constraints 
(for example U({E1,E2})=U(E1)+U(E2) for all E1, E2). More general 
principle(s) need to be worked out that subsumes the SSA as a special 
case.

Reply via email to