Matt Mahoney wrote:
My concern is that structured knowledge is inconsistent with the development of language in children. As I mentioned earlier, natural language has a structure that allows direct training in neural networks using fast, online algorithms such as perceptron learning, rather than slow algorithms with hidden units such as back propagation. Each feature is a linear combination of previously learned features followed by a nonlinear clamping or threshold operation. Working in this fashion, we can represent arbitrarily complex concepts. In a connectionist model, we have, for example:
Pei has already addressed some of the other problems with what you have
said, so I will confine my comments to this part. Perceptron learning
is known (since four decades ago) to have limitations that make it a
ludicrous choice for learning. See Minsky and Papert (the book
"Perceptrons").
And in the sequence of items that follow, some things can be done with
conventional NNs like backprop, but others like phrases and sentences
are completely impossible unless you add something to them.
Your comments that "attempting to parse a sentence first and then
extract its meaning does not work" is naive. Humans clearly do partial
parsing simultaneously with semantic decoding (there is a *huge*
literature on this, but for one choice example, see Frazier, L.,
Clifton, C., & Randall, J. (1983). Filling Gaps: Decision principles
and structure in sentence comprehension. Cognition, 13, 187-222.).
[snip]
Children also learn language as a progression toward increasingly complex patterns.
- phonemes beginning at 2-4 weeks
- phonological rules for segmenting continuous speech at 7-10 months [1]
- words (semantics) beginning at 12 months
- simple sentences (syntax) at 2-3 years
- compound sentences around 5-6 years
ARRrrrrgggghhhh!
Please don't do this. My son (like many other kids) had finished about
fifty small books by the time he was 5, and at least one of the Harry
Potter books when he was 6.
You are talking about these issues at a pre-undergraduate level of
comprehension.
Richard Loosemore.
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/[EMAIL PROTECTED]
