RICHARD LOOSEMORE>>>> I cannot even begin to do justice, here, to the issues involved in solving "the high dimensional problem of seeking to understand the meaning of text, which often involve multiple levels of implication, which would normally be accomplished by some sort of search of a large semantic space"
You talk as if an extension of some current strategy will solve this ... but it is not at all clear that any current strategy for solving this problem actually does scale up to a full solution to the problem. I don't care how many toy examples you come up with, you have to show a strategy for dealing with some of the core issues, AND reasons to believe that those strategies really will work (other than "I find them quite promising"). Not only that, but there at least some people (to wit, myself) who believe there are positive reasons to believe that the current strategies *will* not scale up. ED PORTER>>>> I don't know if you read the Shastri paper I linked to or not, but it shows we do know how to do many of the types of implication which are used in NL. What he shows needs some extensions, so it is more generalized, but it and other known inference schemes explain a lot of how text understanding could be done. With regard to the scaling issue, it is a real issue. But there are multiple reasons to believe the scaling problems can be overcome. Not proofs, Richard, so you are entitled to your doubts. But open your mind to the possibilities they present. They include: -1---------the likely availability of roughly brain level representational, computational, and interconnect capacities within the several hundred thousand to 1 million dollar range in seven to ten years. -2---------the fact that human experience and representation does not explode combinatorially. Instead it is quite finite. It fits insides our heads. Thus, although you are dealing with extremely high dimensional spaces, most of that space is empty. There are know ways to deal with extremely high dimensional spaces while avoiding the exponential explosion made possible by such high dimensionality. Take the well know Growing Neural Gas (GNG) algorithm. It automatically creates a relative compact representation of a possibly infinite dimensional space, by allocated nodes to only those parts of the high dimensional space where there is stuff, or, if resource are more limited, where the most stuff is. Or take indexing, it takes one only to places in the hyperspace where something actually occurred or was thought about. One can have probabilitistically selected hierarchical indexing (something like John Rose suggested) which make indexing much more efficient. -3---------experiential computers focus most learning, most models, and most search on things that actually have happened in the past or on things that in many ways are similar to what has happened in the past. This tends to greatly reduce representational and search spaces. When such a system synthesizes or perceives new patterns that have never happened before the system will normally have to explore large search spaces, but because of the capacity of brain level hardware it will have considerable capability to do so. The type of hardware that will be available for human-level agi in the next decade will probably have sustainable cross sectional bandwidths of 10G to 1T messages/sec with 64Byte payloads/msg. With branching tree activations and the fact that many messages will be regional, the total amount of messaging could well be 100G to 100T such msg/sec. Lets assume our hardware has 10T msg/sec and that we want to read 10 words a second. That would allow 1T msg/word. With a dumb spreading activation rule that would allow you to: active the 30K most probably implications; and for each of them the 3K most probable implications; and for each of them the 300 most probable implications; and for each of them the 30 most probable implications. As dumb as this method of inferencing would be, it actually would make a high percent of the appropriate multi-step inferences, particularly when you consider that the probability of activation at the successive stages would be guided by probabilities from other activations in the current context. Of course there are much more intelligent ways to guide activation that this. Also it is important to understand that at every level in many of the searches or explorations in such a system there will be guidance and limitations provided by similar models from past experience, greatly reducing the amount of or the number of explorations that are required to produce reasonable results. -4---------Michael Collins a few years ago had was many AI researches considered to be the best grammatical parser, which used the kernel trick to effectively match parse trees in, I think it was, 500K dimensions. By use of the Kernel trick the actual computation usually was performed in a small subset of these dimensions and the parser was relatively efficient. -5---------Hecht-Nielsen's sentence completion program (produced by his "confabulation" see http://r.ucsd.edu), just by appropriately tying together probabilistic implications learned from sequences of words, automatically creates grammatically correct sentences that are related to a prior sentense, allegedly without any knowledge of grammar, using millions of probability activations per word, without any un-computable combinatorial explosion. The search space that is being explored at any one time theoretically is considering more possibilities than there are particles in the known universe -- yet it works. At any given time several, lets, say 6 to 12 word or phrase slots can be under computation, in which each of approximately 100K or so words or phrases is receiving scores. One could consider the search space to include each of the possible words or phrase being considered in each of those say 10 ordered slots as the possible permutation of 10 slot fillers each chosen from a set of about 10^5 words or phrases, a permuation that has (10^5)!/(10^4)! possibilities. This is a very large search space -- just 100!/10! is over 10^151¸and (10^5)!/(10^4)! is much, much, much larger space than that -- and yet it all compute with somewhere within several orders of magnitude of a billion opps. This very large search space is actually handled with a superposition of probabilities (somewhat as in quantum computing) which are collapsed in a sequential manner, in a rippling propagation of decisions and ensuing probability propagations. So Richard there are ways to do searches efficiently in very high dimensional spaces, including in the case of confabulation spaces that are in some ways trillions and trillions of times larger than the known universe -- all on relatively small computers. So lift thine eyes up unto Hecht-Nielsen -- (and his cat with whom he generously shares credit for Confabulation) -- and believe! Ed Porter -----Original Message----- From: Richard Loosemore [mailto:[EMAIL PROTECTED] Sent: Monday, December 03, 2007 12:49 PM To: agi@v2.listbox.com Subject: Re: Hacker intelligence level [WAS Re: [agi] Funding AGI research] Ed Porter wrote: > Richard, > > It is false to imply that knowledge of how to draw implications from a > series of statements by some sort of search mechanism is equally unknown as > that of how to make an anti-gravity drive -- if by "anti-gravity drive" you > mean some totally unknown form of physics, rather than just anything, such > as human legs, that can push against gravity. > > It is unfair because there is a fair amount of knowledge about how to draw > implications from sequences of statements. For example view Shastri's > www.icsi.berkeley.edu/~shastri/psfiles/cogsci00.ps. Also Ben Goertzel has > demonstrated a program that draws implications from statements contained in > different medical texts. > > Ed Porter > > P.S., I have enclosed an inexact, but, at least to me, useful drawing I made > of the type of "search" involved in understanding the multiple implications > contained in the series of statements contained in Shastri's "John fell in > the Hallway. Tom had cleaned it. He was hurt" example. Of course, what is > most missing from this drawing are all the other, dead end, implications > which do not provide a likely implication. Only one of such dead end is > shown (the implication between fall and trip). As a result you don't sense > how many dead ends have to be searched to find the implications which best > explain the statements. EWP Well, bear in mind that I was not meaning the analogy to be *that* exact, or I would have given up on AGI long ago - I'm sure you know that I don't believe that getting an understanding system working is as impossible as getting an AG drive built. The purpose of my comment was to point to a huge gap in understanding, and the mistaken strategy of dealing with all the peripheral issues before having a clear idea how to solve the central problem. I cannot even begin to do justice, here, to the issues involved in solving "the high dimensional problem of seeking to understand the meaning of text, which often involve multiple levels of implication, which would normally be accomplished by some sort of search of a large semantic space" You talk as if an extension of some current strategy will solve this ... but it is not at all clear that any current strategy for solving this problem actually does scale up to a full solution to the problem. I don't care how many toy examples you come up with, you have to show a strategy for dealing with some of the core issues, AND reasons to believe that those strategies really will work (other than "I find them quite promising"). Not only that, but there at least some people (to wit, myself) who believe there are positive reasons to believe that the current strategies *will* not scale up. Richard Loosemore > -----Original Message----- > From: Richard Loosemore [mailto:[EMAIL PROTECTED] > Sent: Monday, December 03, 2007 10:07 AM > To: agi@v2.listbox.com > Subject: Re: Hacker intelligence level [WAS Re: [agi] Funding AGI research] > > Ed Porter wrote: >> Once you build up good models for parsing and word sense, then you read >> large amounts of text and start building up model of the realities > described >> and generalizations from them. >> >> Assuming this is a continuation of the discussion of an AGI-at-home P2P >> system, you are going to be very limited by the lack of bandwidth, >> particularly for attacking the high dimensional problem of seeking to >> understand the meaning of text, which often involve multiple levels of >> implication, which would normally be accomplished by some sort of search > of >> a large semantic space, which is going to be difficult with limited >> bandwidth. >> >> But a large amount of text with appropriate parsing and word sense > labeling >> would still provide a valuable aid for web and text search and for many >> forms of automatic learning. And the level of understanding that such a > P2P >> system could derive from reading huge amounts of text could be a valuable >> initial source of one component of world knowledge for use by AGI. > > I know you always find it teious when I express scepticism, so I will > preface my remarks with: take this advice or ignore it, your choice. > > This description of how to get AGI done reminds me of my childhood > project to build a Mars-bound spacecraft modeled after James Blish's > Book "Welcome to Mars". I Knew that I could build it in time for the > next conjunction of Mars, but I hadn't quite gotten the anti-gravity > drive sorted out, so instead I collected all the other materials > described in the book, so everything would be ready when the AG drive > started working... > > The reason it reminds me of this episode is that you are calmly talking > here about "the high dimensional problem of seeking to understand the > meaning of text, which often involve multiple levels of implication, > which would normally be accomplished by some sort of search of a large > semantic space" ......... this is your equivalent of the anti-gravity > drive. This is the part that needs extremely detailed knowledge of AI > and psychology, just to be understand the nature of the problem (never > mind to solve it). If you had any idea bout how to solve this part of > the problem, everything else would drop into your lap. You wouldn't > need a P2P AGI-at-home system, because with this solution in hand you > would have people beating down your door to give you a supercomputer. > > Menawhile, unfortunately, solving all those other issues like making > parsers and trying to do word-sense disambiguation would not help one > whit to get the real theoretical task done. > > I am not being negative, I am just relaying the standard understanding > of priorities in the AGI field as a whole. Send complaints addressed to > "AGI Community", not to me, please. > > > > Richard Loosemore > > > ----- > This list is sponsored by AGIRI: http://www.agiri.org/email > To unsubscribe or change your options, please go to: > http://v2.listbox.com/member/?&; > > ----- > This list is sponsored by AGIRI: http://www.agiri.org/email > To unsubscribe or change your options, please go to: > http://v2.listbox.com/member/?&; > > > ------------------------------------------------------------------------ > ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?&; ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=71673866-9b97ad
<<attachment: winmail.dat>>
