--- "YKY (Yan King Yin)" <[EMAIL PROTECTED]> wrote: > I have an intuition about language learning... > > There're 2 different approaches: > > A. Learning like a human baby. Start with single words, and then proceed to > simple sentences, and so on. Each successive "layer" building on the > foundation of lower layers. > > B. Learning directly from "adult" text corpuses, ie going directly from age > 0 to age 6. > > My intuition is that B requires exponentially more computation than A. In > other words, the "layers-based" learning pathway reduces computational > complexity *logarithmically*. But I don't know how to prove it. Does it > make sense? Can someone corroborate? > > If this is correct, then we must start with simple sentences (Basic English) > and not with mining the web or newspapers.
I think learning in layers (A) is the correct approach, but also that it can be done from a corpus of adult level language, at least if you are training a pure, ungrounded language model. When parents use baby talk, they are actually using fragments of adult level language, like holding a ball and saying "ball". In an ungrounded model (no vision, robotics, etc), the model only needs to know that "ball" is likely to appear in immediate contexts like "a" or "the" and more distant contexts like "play" or "catch". It doesn't need to know what these words mean in the same sense that we do. Parents might use baby talk like "goo goo ga ga" but I don't think that helps children learn language. It might help them learn to associate their own utterances with what they hear, but you don't need that type of training for a statistical model trained on text. I know you can't completely solve any of the AI problems like language, vision, hearing, or robotics without solving all of them, but I do think it is possible to induce ungrounded semantics (word associations) and grammar in a statistical model without having to train on children's books. -- Matt Mahoney, [EMAIL PROTECTED] ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=231415&user_secret=fabd7936
