That's one of my personal areas of uncertainty, looking in on artificial intelligence and cognitive science. We each as humans make our final judgements based on intuition at the deepest level, where philosophy collapses against the weight of permanent uncertainty. How do you propose (addressing everyone) to overcome the tendency that our AGIs will have to emulate the intuitive, non-rational preferences that we unconsciously transfer to them? Where do you think the AGI will get its own personal preference to think in a way separate from its masters?
How will rigorous symbolic decisionmaking eventually break through the philosophical wall of uncertainty, in your model?
It won't. Uncertainty is a necessary part of life in the real world; formal logic just isn't a powerful enough tool by itself to deal with it.
Our intuitions have evolved over millions of years of testing; while necessarily imperfect, they're proven powerful enough for survival, at least. I expect any successful AGI will have to adopt those intuitions or something similar to them, at least for awhile. Ultimately it may start developing its own, better intuitions; but I see that as something for the far future, beyond our prediction horizon.
(An aside on the original topic: if you look at Marc's posts for the last few years, the main thing wrong with them has been too much drift into philosophy and intuition away from hard data and formal reasoning; so in fairness, this switch to overemphasis on rigor, while not absolutely true, may be something he needs to do at the moment.)
To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
