On Thu, Jun 12, 2008 at 3:36 AM, Steve Richfield <[EMAIL PROTECTED]> wrote: > ... and here we have the makings of AGI run amok... > My point.. it is usually possible to make EVERYONE happy with the results, > but only with a process that roots out the commonly held invalid assumptions. > Like Gort (the very first movie AGI?) in The Day The Earth Stood Still, the > goal is peace, but NOT through any particular set of detailed goals.
I think it's important to distinguish between supervised and unsupervised AGIs. For the supervised, top-level golas as well as the sub-goal restrictions can be volatile - basically whatever the guy in charge wants ATM (not neccessarily trying to make EVERYONE happy). In that case, AGI should IMO just attempt to find the simplest solution to a given problem while following the given rules, without exercising its own sense of morality (assuming it even has one). The guy (/subject) in charge is the god who should use his own sense of good/bad/safe/unsafe, produce the rules to follow during AGI's solution search and judge/approve/reject the solution so he is the one who bears responsibility for the outcome. He also maintains the rules for what the AGI can/cannot do for "lower-level" users (if any). Such AGIs will IMO be around for a while. *Much* later, we might go for human-unsupervised AGIs. I suspect that at that time (if it ever happens), people's goals/needs/desires will be a lot more unified/compatible (so putting together some grand schema for goals/rules/morality will be more straight forward) and the AGIs (as well as its multi-layer and probably highly-redundant security controls) will be extremely well tested = highly unlikely to "run amok" and probably much safer than the previous human-factor-plagued problem solving hybrid-solutions. People are more interested in pleasure than in messing with terribly complicated problems. Regards, Jiri Jelinek *** Problems for AIs, work for robots, feelings for us. *** ------------------------------------------- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=103754539-40ed26 Powered by Listbox: http://www.listbox.com
