Stathis Papaioannou wrote:
Jesse Mazer writes:
Jesse Mazer writes (after quoting Stathis Papaioannou):
No doubt, common implementations of your mind will predominate over more bizarre ones at any given point in time. It is also possible to imagine some scenarios where you survive indefinitely with all of your friends, for example implemented in an Omega Point computer. But eternity is a very long time. If it is possible that the Omega Point computer can break down, then, as Murphy teaches, it certainly *will* break down - eventually.
Not if the probability of it breaking down decreases in a geometric way from century to century (or millennium to millennium, aeon to aeon, whatever) as more and more of the universe is incorporated into the giant distributed computing network (or as the increasing computing power allows for more and more sophisticated ways of anticipating and avoiding civilization-ending disasters). Like I said, if the probability of a catastrophic breakdown was 1/8 in one century, 1/16 in the next, 1/32 in the next, and so on, then the total probability of it breaking down at any point in the entire infinite history of the universe would be the sum of the infinite series 1/8+1/16+1/32+1/64+1/128+... , which is equal to 1/4. In such a branch there'd be a 3/4 chance that civilization would last forever.
It is possible that the probability of the computer breaking down decreases geometrically with time, as you say. However, as t->infinity, it is nevertheless increasingly likely to deviate from this ideal behaviour, and the measure of branches of the multiverse in which it does will approach zero. Remember, it is not the probability in any single branch which is important (in fact, in the MWI that would be a meaningless concept), but the measure across all branches.
It may be more likely to deviate from this ideal behavior, but it could deviate by approaching zero probability of breakdown faster than the ideal behavior predicts, instead of slower; when I said that the probability would be 1/8+1/16+1/32+..., I meant the *average* you get when you sum all possible future histories from that point, including both the histories where at some later time the probability was approaching zero even faster than predicted by the 1/8+1/16+... pattern along with the histories where at some later time it was approaching zero slower, or the probability of breakdown was even increasing. Since it's an average, that means that out of all future histories stemming from that time, in 3/4 of them civilization will never break down.
There are two separate probabilities to consider here. One is the probability (3/4, as you show) that civilization will never break down if implemented on a computer with behaviour as specified above. The other is the probability that the actual hardware will work according to specification. I don't think you should conflate the two, effectively arguing that the hardware will work to specification because that is part of the specification!
I don't think I ever said anything about the probability involving software only. If you have a distributed computing network (such that destroying any part of it won't cause a global breakdown), and more and more of the universe is constantly being gobbled up and converted into computing power, then perhaps the probability of all the hardware in the universe breaking down would decrease geometrically as well, on average. Assume that when I talk about the probability of all copies of you being destroyed decreasing like 1/8+1/16+1/32+..., this probability takes into account all possible causes of failure, including software problems, destruction of hardware, and even stuff like the possibility that some other enemy groups of A.I.'s will attempt to erase all copies of you.
Returning to the original question, once you have settled into your new home, what is to stop all your friends disappearing, as before? The computer will try to prevent this from happening, and you could probably try the geometric series trick again (i.e. decreasing probability that your friends disappear), but in this case there will be nothing tying you to those ever-rarer branches where the hardware works as it is supposed to.
But my point is that it doesn't necessarily have to be a matter of "ever-rarer" branches--even aside from quantum immortality, it might be true that in 3/4 (or whatever) of all branches stemming from a given point in time, any A.I. around at that time will have at least some copies around in the giant computing network forever.