Richard Loosemore writes:> You must remember that the complexity is not a massive part of the > system, just a small-but-indispensible part.> > I think this sometimes causes confusion: did you think that I meant > that the whole thing would be so opaque that I could not understand > *anything* about the behavior of the system? Like, all the > characteristics of the system would be one huge emergent property, with > us having no idea about where the intelligence came from? No, certainly not. I think the confusion here involves the distinction between Friendliness with a capital F (meaning a formal theory of what the term means and an intelligent system built to provably maintain that property in the mathematical, not verbal, sense), and friendliness with a lower case f, which relies on more human types of reasoning. It sounds to me like on the one hand you are saying that your system is complex and yet its behavior is not complex (at least in a particular but quite broad way - friendliness), as if you can bottle up the complexity in such a way that it has no important actual effects. In particular, when you write: > You can build such a motivational system by controlling the system's > agenda > by diffuse connections into the thinking component that controls > what it > wants to do. It seems like doing that requires a rigorous understanding of the dynamics of the thinking component and I don't quite get how that can work in a "guaranteed" way since elements of the thinking component may change their nature in unpredictable ways and new elements of the thinking component may arise. If the nature of all thinking component elements are indeed perfectly predictable so that is impossible, I don't get where the complexity is. In your AGIRI paper, you seem to be saying that intelligence itself is an emergent property of a complex system (sorry to use words that everybody is probably allergic to), which seems to imply a global impact of the underlying complexity. I think (probably incorrectly) that I have a rough idea of how you intend to guide this complex system, and in fact I think that is likely the only way to go about it. It makes me nervous a bit when phrasings that bring capital-F Friendliness to mind are applied to designs that cannot possibly exhibit it. Basically, it rings alarm bells about overconfidence. However, the details of what you actually are working on will explain more than this conversation can, I'm sure.
----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=48433850-805862
