--- On Fri, 10/24/08, Russell Wallace <[EMAIL PROTECTED]> wrote: > On Fri, Oct 24, 2008 at 3:48 PM, Matt Mahoney > <[EMAIL PROTECTED]> wrote: > > This will be practical once we have a million-fold > > decrease in the cost of computation, based on the cost of > > simulating a brain sized neural network. It could occur > > sooner if we discover more efficient > > solutions. So far we have not. > > Nor have I stopped looking :-)
Nor should we. > > If you are selecting a language for self improving AI > using a genetic algorithm > > Oh, I gave up on that approach years ago, what with wanting > a solution before the sun burns out and all. That's why you need a fault tolerant language that works well with redundancy. However you still have the inherent limitation that genetic algorithms can learn no faster than 1 bit per population doubling. > > If you are selecting a language to implement language > or vision, good choices are C, C++, and assembler. The > primary concern is efficiency and the ability to make good > use of underlying parallelism in the hardware. The choice > will probably be less important as hardware gets faster. > > The above are reasonable choices for low level pixel > crunching, if > that's all your project is doing. None of them is > remotely suitable > for implementing anything related to language. That is not all I am doing. Look at the top ranked text compressors. They implement fairly sophisticated language models, though still far below adult level. There are essentially 3 mechanisms involved in learning language and vision: 1. eidetic (short term) memory, 2. learning associations between concepts in short term memory, 3. clustering in context space to form new concepts. All three have straightforward neural implementations. In language, the concepts are phonemes, words, parts of speech, and grammatical structures. In vision, they are recognizable and distinct objects such as lines or faces. There is a learning hierarchy in which complex features are associated with simpler features previously learned. An example of clustering is when you learn a new type by generalization, e.g. "the dog ate", "the cat ate", "Bob ate", etc. to form a new category (things that eat). In the top ranked compressor (durilca4linux), the dictionary was organized using this type of clustering. It allows related words to be treated as a shared context for predicting the next word. Other top ranked compressors such as paq8hp12 use distant bigram contexts to model semantics. This type of computation is expensive because a typical human vocabulary has 10^5 concepts. A lot of people have looked for efficient solutions to the natural language modeling problem. The two graphs below the main table at http://cs.fit.edu/~mmahoney/compression/text.html clearly show that CPU speed and memory are severe constraints, regardless of the modeling technique. -- Matt Mahoney, [EMAIL PROTECTED] ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=117534816-b15a34 Powered by Listbox: http://www.listbox.com
