On Tue, Aug 26, 2008 at 11:13 PM, Valentina Poletti <[EMAIL PROTECTED]> wrote: > Vlad, Terren and all, > > by reading your interesting discussion, this saying popped in my mind.. > admittedly it has little to do with AGI but you might get the point anyhow: > > An old lady used to walk down a street everyday, and on a tree by that > street a bird sang beautifully, the sound made her happy and cheerful and > she was very thankful for that. One day she decided to catch the bird and > place it into a cage, so she could always have it singing for her. > Unfortunately for her, the bird got sad in the cage and stopped singing... > thus taking away her cheer as well. > > Well, the story has a different purpose, but one can see a moral that > connects to this argument. Control is an illusion. It takes away the very > nature of what we are trying to control.
Then you are doing something wrong. The natural word "control" biases how you think about this issue, creating associations with caged birds, imprisonment, shattered potential and stupid mechanical robots. Think instead of determination, lawfulness and rational decisions. You do not see yourself as being controlled, as being limited in your ability to e.g. eat human babies. Control embodied in you that prevents you from doing that doesn't take away your human nature; on the contrary, it is a part of human nature. What is genetically determined in humans is not there to constrain us, it is not inflexible and fixed as opposed to general ability of intelligence. Instead, it is what enables us to be flexible and generally intelligent, to see what is good. We are determined and controlled by our nature, but we don't want to escape it, instead we want to improve on it from within (see http://www.overcomingbias.com/2008/07/rebelling-withi.html ). For more about how freedom comes as lawfulness, in intricate and open-ended forms, see Tooby and Cosmides "The psychological foundations of culture" ( http://folk.uio.no/rickyh/papers/TheAdaptedMind.htm ). Needing to know what we are doing is *necessary* to avoid ruin. You can't create a Friendly AI with 60% of success, and you can't create an FAI with 1% of success, because it is too hard to know how likely it is to work. If you can create an FAI and know that it has 1% chance to work, you understand FAI well enough to make one that is almost guaranteed to work. And if you don't understand it well enough to say that it has that 1% chance of succeeding, how do you know that it's not in fact a lottery ticket that you have no hope of winning? The question of Friendly AI has some amount of complexity, and unless you know what you are doing, you will be confronted by this complexity playing against you, exponentially reducing your chances. You can't hope to hit a narrow target being blindfolded and boasting that you have a chance. Even when you see the target, you probably won't be ready and will need to continue working on your skill instead. -- Vladimir Nesov [EMAIL PROTECTED] http://causalityrelay.wordpress.com/ ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=111637683-c8fa51 Powered by Listbox: http://www.listbox.com
