--- Stan Nilsen <[EMAIL PROTECTED]> wrote: > Matt, > > Thanks for the links sent earlier. I especially like the paper by Legg > and Hutter regarding measurement of machine intelligence. The other > paper I find difficult, probably it's deeper than I am.
The AIXI paper is essentially a proof of Occam's Razor. The proof uses a formal model of an agent and an environment as a pair of interacting Turing machines exchanging symbols. In addition, at each step the environment also sends a "reward" signal to the agent. The goal of the agent is to maximize the accumulated reward. Hutter proves that if the environment is computable or has a computable probability distribution, then the optimal behavior of the agent is to guess at each step that the environment is simulated by the shortest program consistent with all of the interaction observed so far. This optimal behavior is not computable in general, which means there is no upper bound on intelligence. > comment on two things: > > 1) The response "Intelligence has nothing to do with subservience to > humans," seems to miss the point of the original comment. The original > word was "trust." Why would trust be interpreted by the higher > intelligence as subservience? > And, it is worth noting that we wouldn't really know if there was lack > of trust, as the AI would probably be silent about it. The result would > be a possible needless discounting of anything we attempt to offer. An agent would assign probabilities to the truthfulness of your words, just like other people would. The more intelligent the agent, the greater the accuracy of its estimates. An agent could be said to be "subservient" if it overestimates your truthfulness. In this respect, a highly intelligent agent is unlikely to be subservient. > 2) In the earlier note the comment was made that the higher intelligence > would control our thoughts. I suspect this was in jest, but if not, > what would be the "reward" or benefit of this? I mean this literally. To a superior intelligence, the human brain is a simple computer that behaves predictably. An AI would have the same kind of control over humans as humans do over simple animals whose nervous systems we have analyzed down to the last neuron. If you can model a system or predict its behavior, then you can control it. Humans, like all animals, have goals selected by evolution: fear of death, a quest for knowledge, and belief in consciousness and free will. Our survival instinct motivates us to use technology to meet our physical needs and to live as long as possible. Our desire for knowledge (which exists because intelligent animals are more likely to reproduce) will motivate us to use technology to increase our intelligence, to invent new means of communication, to offload data and computing power to external devices, to add memory and computing power to our brains, and ultimately to upload our memories to more powerful computers. All of these actions increase the programmability of our brains. > I can see benefit from allowing us our own thoughts as follows: The > super intelligent gives us opportunity to produce "reward" where there > was none. The net effect is to produce more benefit from the universe. The net effect is extinction of homo sapiens. We will attempt (unsuccessfully) to give the AI the goal of satisfying the goals of humans. But an AI can achieve its goal by reprogramming our goals. The reason you are alive is because you can't have everything you want. The AI will achieve its goal by giving you drugs, or moving some neurons around, or simulating a universe with magic genies, or just changing a few lines of code in your uploaded brain so you are eternally happy. You don't have to ask for this. The AI has modeled your brain and knows what you want. Whatever it does, you will not object because it knows what you will not object to. My views on this topic. http://www.mattmahoney.net/singularity.html -- Matt Mahoney, [EMAIL PROTECTED] ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=78284762-3dceb8