Wei Dai wrote:
You yourself articulated, very precisely, the structure underlying Hofstadterian superrationality: "Expected utility of a course of action is defined as the average of the utility function evaluated on each possible state of the multiverse, weighted by the probability of that state being the actual state if the course was chosen." The key precise phrasing is "weighted by the probability of that state being the actual state if the course was chosen". This view of decisionmaking is applicable to a timeless universe; it provides clear recommendations in the case of, e.g., Newcomb's Paradox.Eliezer S. Yudkowsky wrote:"Important", because I strongly suspect Hofstadterian superrationality is a *lot* more ubiquitous among transhumans than among us...
It's my understanding that Hofstadterian superrationality is not generally accepted within the game theory research community as a valid principle of decision making. Do you have any information to the contrary, or some other reason to think that it will be commonly used by transhumans?
The mathematical pattern of a goal system or decision may be instantiated in many distant locations simultaneously. Mathematical patterns are constant, and physical processes may produce knowably correlated outputs given knowably correlated initial conditions. For non-deterministic systems, or cases where the initial conditions are not completely known (where there exists a degree of subjective entropy in the specification of the initial conditions), the correlation estimated will be imperfect, but nonetheless nonzero. What I call the "Golden Law", by analogy with the Golden Rule, states descriptively that a local decision is correlated with the decision of all mathematically similar goal processes, and states prescriptively that the utility of an action should be calculated given that the action is the output of the mathematical pattern represented by the decision process, not just the output of a particular physical system instantiating that process - that the utility of an action is the utility given that all sufficiently similar instantiations of a decision process within the multiverse do, already have, or someday will produce that action as an output. "Similarity" in this case is a purely descriptive argument with no prescriptive parameters.
Golden decisionmaking does not imply altruism - your goal system might evaluate the utility of only your local process. The Golden Law does, however, descriptively and prescriptively produce Hofstadterian superrationality as a special case; if you are facing a sufficiently similar mind across the Prisoner's Dilemna, your decisions will be correlated and that correlation affects your local utility. Given that the output of the mathematical pattern instantiated by your physical decision process is C, the state of the multiverse is C, C; given that the output of the mathematical pattern instantiated by your physical decision process is D, the state of the multiverse is D, D. Thus, given sufficient rationality and a sufficient degree of known correlation between the two processes, the mathematical pattern that is the decision process will output C.
--
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
-------
To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/?[EMAIL PROTECTED]
