Ben:
I have no near-term plans to open-source Novamente. I think that
would be a bad idea for AGI safety reasons. I am worried that if we
truly succeed in making a human-level intelligence, and opened up the
code, some jerks might do really nasty things with it.
[ It's good that you raised these issues. We need to practice on thinking about them. I'll put this stuff on my web pages later. ]
Your argument may also backfire against you. If you think the general public cannot be entrusted to the use of AGI, then the business sector should not be exempt from this rule. This way of thinking will lead to complete banning of AGI from the public sector, with the government being the only body allowed to build AGIs, appointing special programmer screened for moral superiority. Which seems very totalitarian. Not to mention that this would be difficult, I mean almost impossible, to enforce.
Perhaps we should accept the fact that some people may put AGI to malicious usage. Just as e-mail has enabled the use of spam and we developed anti-spam to counter it, similarly we would develop counter measures for malicious AGI.
Government regulation of AGI-building is unlikely to occur, because:
1. It is difficult to enforce;
2. Other countries that don't enforce such a rule may get an advantage;
3. It infringes on personal freedom -- AGI is not necessarily harmful to others.
Government regulation of AGI usage may be possible. For example, there can be a monitoring module that triggers an alarm when suspicious commands are given to the AGI. Although this is not 100% secure, it is a practical solution.
AGI should not be analogous to dynamite, guns, or nuclear energy, because:
1. Dynamite has 2 main uses: as weapon and as construction tool, the latter is allowed under regulation;
2. Gun has one main use: as weapon, which is why it is banned in many countries, and one lesser use: as recreation, which is allowed under regulation;
3. Nuclear energy has 2 main uses: as mass destruction weapon and as energy. The former use is banned, the latter use is restricted because the general public has no use for nuclear energy at small scales;
4. AGI is *mainly* a useful civilian tool with a possibility of becoming malicious. AGI is different from 1-3 in the sense that it is NOT primarily a weapon and that it is tremendously useful in the civilian sector. According to this trend it is likely that AGI will be allowed under regulations.
Publishing the book is also a safety risk but less so because
replicating the code from a fairly abstract book would be pretty hard.
It's sorta like the difference between publishing a text on nuclear
physics versus publishing the exact specifications for a nuclear
bomb...
Again, your argument would cause the entire private sector to lose the right to build AGIs. AGI building would then be the job of government appointed specialists.
yky
To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
