Hi Fergal, Let me stand up for the AI. In my opinion a lot more done here than in NN. For example, a hierarchical analysis of scenes based on models and the corresponding inductive-deductive learning process and analysis. Much like the HTM. Your concepts were born in AI.
On the other hand, how far you've gone from summer in the modeling of a neuron? Your scheme of the brain in the same distance. Distance from the word "cat" to the word "dog" is defined by the evolution of language. This distance corresponds to the semantic relations of cats and dogs. When you add semantic information to the morphological level you make those words much closer. You only change the pattern of the word, but its meaning will be determined by the texts or other modalities. Sorry to bother you. I have some experience in NN. Now I've done a first-order logic as an example NLP with fast learning. My NN fits into the part of the cortex column. Of course, I followed the publications of Jeff and I was looking at your experience with the columns. My opinion of CEPT related to my study of its use in my NN. Method CEPT is well done, but it should be inside the NN system. I just wanted to share that opinion. Regards, Ivan ON:Date: Sun, 15 Sep 2013 13:08:34 +0100 From: Fergal Byrne <[email protected]> To: "NuPIC general mailing list." <[email protected]> Subject: Re: [nupic-dev] (no subject) Message-ID: <cad2q5yex+jr28sbx9tutubzgnad1jhdpv8zr1oy8c31xu0o...@mail.gmail.com> Content-Type: text/plain; charset="iso-8859-1" Hi Ivan, I'm not sure what you're driving at here, but let me just try to answer your concerns. While a huge amount of (largely fruitless IMHO) work has been done in classical AI research on NLP, this work has a fatal flaw. It is looking at the problem as one which can be solved by combining ever more complex programs in ever more complex ways. The only good thing it has achieved is that it has asked a lot of questions about the structure of language, and also allowed computers to analyse huge amounts of text in order to get good statistical understandings of what we actually say. Loads of clever software cannot be the answer. Only one system in the universe has been built which does NLP, and that system uses no programs. It has some inbuilt structure, but it's essentially a set of simple CLA's wired up in a particular way. So, what we're attempting to do here is to assess what happens when you look at one (hypothetical) piece of this system. All regions in our language systems communicate by passing SDR's up the hierarchy. So we've got some of these from CEPT and we're going to do some experiments to see can we observe anything similar to what we see in the brain. The classifiers we use are just implementation artefacts which allow us to measure and observe the activation patterns and predictions that the CLA is generating. These classifiers are like electrode arrays or fMRI pictures, and are not part of the simulation. We aren't going to be writing classifiers to perform the tasks you listed - in fact the opposite is the case. We plan to build hierarchies of CLA's which perform those tasks, with only very simple (but massively parallel) plumbing between them providing the "cleverness". The key to figuring out how the brain achieves all these wonderful feats is to figure out how to connect the regions and combine them together in the hierarchy. The research begins with this simple question: How do we (or just can we) connect the CEPT SDR's, representing words, to a "word-level" region of CLA, in order to produce some meaningful linguistic performance? The answer will be dependent on both a) whether the CEPT data is a good analogue for how words are passed around in the brain, and b) whether we can design a way to connect up the CEPT SDR's to the CLA which works. We know neither of these for sure, but by varying the connection strategies, we should be able to provide an answer to the former question. This should allow us (if we fail) to improve the CEPT data and try again. It is possible that we're barking up the wrong tree with the CEPT SDR's. If so, we should find out as soon as we can, and then see if a pivot will yield fruit. Humans are doing something like this, so let's see if we can generate the appropriate analogue with just a single region, or if we need to add hierarchy to get there. Regards Fergal Byrne
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
