On Friday, January 17, 2014 5:14:13 AM UTC-6, Bruno Marchal wrote:
> To be franc, I don't believe in super-intelligence. I do believe in
> super-competence, relative to some domain, but as I have explained from
> time to time, competence has a negative feedback on intelligence.
> Intelligence is a state of mind, almost only an attitude. Some animals are
"Intelligence" is one of those big broad words that can be taken different
ways. The MIRI folk are operating under a very specific notion of it. In
making an AI, they primarily want to make a machine that follows the
optimal decision theoretic approach to maximizing its programmed utility
function, and that continues to follow the same utility function even when
it's allowed to change its own code. They don't mean that it has to be
conscious or self-aware or a person or thoughtful or extraordinarily
perceptive or able to question its goals or so on.
Given that approach, then there are utility functions that would be totally
disastrous for humanity, and there may be some that turn out very good for
humanity. So the question of "friendliness" is how best to build an AI
with a utility function that is good for humanity and would stay good for
humanity even as the AI rewrote its own software.
You received this message because you are subscribed to the Google Groups
"Everything List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
To post to this group, send email to email@example.com.
Visit this group at http://groups.google.com/group/everything-list.
For more options, visit https://groups.google.com/groups/opt_out.