Vlad, Terren and all, by reading your interesting discussion, this saying popped in my mind.. admittedly it has little to do with AGI but you might get the point anyhow:
An old lady used to walk down a street everyday, and on a tree by that street a bird sang beautifully, the sound made her happy and cheerful and she was very thankful for that. One day she decided to catch the bird and place it into a cage, so she could always have it singing for her. Unfortunately for her, the bird got sad in the cage and stopped singing... thus taking away her cheer as well. Well, the story has a different purpose, but one can see a moral that connects to this argument. Control is an illusion. It takes away the very nature of what we are trying to control. My point is that by following Eliezer's approach we might never get an AGI. Intelligence, as I defined, is the ability to reach goals and those goals (as Terren pinted out) must somehow have to do with self-preservation if the system itself is not at equilibrium. Not long ago, I was implementing the model for a bio-physics research in New York based on non-linear systems far from equilibrium and this became pretty evident to me. If any system is kept far from equilibrium, it has the tendency to form self-preserving entities, given enough time. I also have a nice definition of friendliness: firstly note that we have goals both as individuals and as a species (these goals are inherent in the species and come from self-preservation -see above). The goals of the individuals and species must somehow match, i.e. not be in significant conflict, least the individual be considered a criminal, harmful. By friendliness is meant creating an AGI which will follow the goals of the human species rather than its own goals. Valentina ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=111637683-c8fa51 Powered by Listbox: http://www.listbox.com
