YKY, I am not proposing any restriction of anyone's legal rights.
I am merely discussing what I believe is the most ethical action for me to take, as a person leading an AGI project. If I developed a new chemical substance usable as an extremely and unprecedentedly dangerous weapon, I would likely not release the plans. But nor would I advocate the banning of chemical research. The issue of long-term AGI ethics and AGI Friendliness is a really thorny one, in so many ways. It's hard to prove whether one action or another is truly better. But we need to make decisions, and in doing so we rely our our intuitions, and my intuition based on the information available is that it's wisest not to release the Novamente source code publicly. The *current* Novamente source code, plus the Novamente book and other documents, wouldn't motivate nasty terrorists to create an AGI at this point, even if openly released. There's still a lot of work to be done on NM, and very few have the capability to do it --and the book and code at this stage, frankly, are not going to convince very many people of the long-term viability of the project. However, looking forward, let's suppose that in 5-10 years the NM project succeeds in creating a massively powerful AGI. At that point, nasty terrorists might well decide it was worth trying to create their own AGI -- and then they could look back at the old NM code and NM book from 2005 and it would give them valuable clues, which they *would* have motivation to follow up upon , due the demonstrated success of the 2010 or 2015 or whatever version of NM. I see no need to take that risk, given that in my view the benefit from opening up the NM source code would be pretty small. NM, at this stage, does not need the scattered efforts of dozens of folks, it needs the full-time concentrated effort of a small and dedicated team. -- Ben G On 12/10/05, Yan King Yin <[EMAIL PROTECTED]> wrote: > > Ben: > > I have no near-term plans to open-source Novamente. I think that > > would be a bad idea for AGI safety reasons. I am worried that if we > > truly succeed in making a human-level intelligence, and opened up the > > code, some jerks might do really nasty things with it. > > [ It's good that you raised these issues. We need to practice on thinking > about them. I'll put this stuff on my web pages later. ] > > Your argument may also backfire against you. If you think the general > public cannot be entrusted to the use of AGI, then the business sector > should not be exempt from this rule. This way of thinking will lead to > complete banning of AGI from the public sector, with the government being > the only body allowed to build AGIs, appointing special programmer screened > for moral superiority. Which seems very totalitarian. Not to mention that > this would be difficult, I mean almost impossible, to enforce. > > Perhaps we should accept the fact that some people may put AGI to malicious > usage. Just as e-mail has enabled the use of spam and we developed > anti-spam to counter it, similarly we would develop counter measures for > malicious AGI. > > Government regulation of AGI-building is unlikely to occur, because: > 1. It is difficult to enforce; > 2. Other countries that don't enforce such a rule may get an advantage; > 3. It infringes on personal freedom -- AGI is not necessarily harmful to > others. > > Government regulation of AGI usage may be possible. For example, there can > be a monitoring module that triggers an alarm when suspicious commands are > given to the AGI. Although this is not 100% secure, it is a practical > solution. > > AGI should not be analogous to dynamite, guns, or nuclear energy, because: > 1. Dynamite has 2 main uses: as weapon and as construction tool, the latter > is allowed under regulation; > 2. Gun has one main use: as weapon, which is why it is banned in many > countries, and one lesser use: as recreation, which is allowed under > regulation; > 3. Nuclear energy has 2 main uses: as mass destruction weapon and as > energy. The former use is banned, the latter use is restricted because the > general public has no use for nuclear energy at small scales; > 4. AGI is *mainly* a useful civilian tool with a possibility of becoming > malicious. AGI is different from 1-3 in the sense that it is NOT primarily > a weapon and that it is tremendously useful in the civilian sector. > According to this trend it is likely that AGI will be allowed under > regulations. > > > > > Publishing the book is also a safety risk but less so because > > replicating the code from a fairly abstract book would be pretty hard. > > It's sorta like the difference between publishing a text on nuclear > > physics versus publishing the exact specifications for a nuclear > > bomb... > > Again, your argument would cause the entire private sector to lose the right > to build AGIs. AGI building would then be the job of government appointed > specialists. > > yky > > > ________________________________ > To unsubscribe, change your address, or temporarily deactivate your > subscription, please go to > http://v2.listbox.com/member/[EMAIL PROTECTED] > > ------- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
