--- Richard Loosemore <[EMAIL PROTECTED]> wrote: > Derek Zahn wrote: > > Richard Loosemore writes: > > > > > It is much less opaque. > > > > > > I have argued that this is the ONLY way that I know of to ensure that > > > AGI is done in a way that allows safety/friendliness to be guaranteed. > > > > > > I will have more to say about that tomorrow, when I hope to make an > > > announcement. > > > > Cool. I'm sure I'm not the only one eager to see how you can guarantee > > (read: prove) such specific detailed things about the behaviors of a > > complex system. > > Hmmm... do I detect some skepticism? ;-)
I remain skeptical. Your argument applies to an AGI not modifying its own motivational system. It does not apply to an AGI making modified copies of itself. In fact you say: > Also, during the development of the first true AI, we would monitor the > connections going from motivational system to thinking system. It would > be easy to set up alarm bells if certain kinds of thoughts started to > take hold -- just do it by associating with certain keys sets of > concepts and keywords. While we are designing a stable motivational > system, we can watch exactly what goes on, and keep tweeking until it > gets to a point where it is clearly not going to get out of the large > potential well. You refer to the humans building the first AGI. Humans, being imperfect, might not get the algorithm for friendliness exactly right in the first iteration. So it will be up to the AGI to tweak the second copy a little more (according to the first AGI's interpretation of friendliness). And so on. So the goal drifts a little with each iteration. And we have no control over which way it drifts. -- Matt Mahoney, [EMAIL PROTECTED] ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=48339385-0a1b82
