Ed Porter wrote:
RICHARD LOOSEMORE>>>> I cannot even begin to do justice, here, to the issues
involved in solving "the high dimensional problem of seeking to understand
the meaning of text, which often involve multiple levels of implication,
which would normally be accomplished by some sort of search of a large
semantic space"
You talk as if an extension of some current strategy will solve this ... but
it is not at all clear that any current strategy for solving this
problem actually does scale up to a full solution to the problem. I don't
care how many toy examples you come up with, you have to show a
strategy for dealing with some of the core issues, AND reasons to believe
that those strategies really will work (other than "I find them
quite promising").
Not only that, but there at least some people (to wit, myself) who believe
there are positive reasons to believe that the current
strategies *will* not scale up.
ED PORTER>>>> I don't know if you read the Shastri paper I linked to or
not, but it shows we do know how to do many of the types of implication
which are used in NL. What he shows needs some extensions, so it is more
generalized, but it and other known inference schemes explain a lot of how
text understanding could be done.
With regard to the scaling issue, it is a real issue. But there are
multiple reasons to believe the scaling problems can be overcome. Not
proofs, Richard, so you are entitled to your doubts. But open your mind to
the possibilities they present. They include:
-1---------the likely availability of roughly brain level representational,
computational, and interconnect capacities within the several hundred
thousand to 1 million dollar range in seven to ten years.
-2---------the fact that human experience and representation does not
explode combinatorially. Instead it is quite finite. It fits insides our
heads.
Thus, although you are dealing with extremely high dimensional spaces, most
of that space is empty. There are know ways to deal with extremely high
dimensional spaces while avoiding the exponential explosion made possible by
such high dimensionality.
Take the well know Growing Neural Gas (GNG) algorithm. It automatically
creates a relative compact representation of a possibly infinite dimensional
space, by allocated nodes to only those parts of the high dimensional space
where there is stuff, or, if resource are more limited, where the most stuff
is.
Or take indexing, it takes one only to places in the hyperspace where
something actually occurred or was thought about. One can have
probabilitistically selected hierarchical indexing (something like John Rose
suggested) which make indexing much more efficient.
I'm sorry, but this is not addressing the actual issues involved.
You are implicitly assuming a certain framework for solving the problem
of representing knowledge ... and then all your discussion is about
whether or not it is feasible to implement that framework (to overcome
various issues to do with searches that have to be done within that
framework).
But I am not challenging the implementation issues, I am challenging the
viability of the framework itself.
My mind is completely open. But right now I raised one issue, and this
is not answered.
I am talking about issues that could prevent that framework from ever
working no matter how much computing power is available.
You must be able to see this: you are familiar with the fact that it is
possible to frame a solution to certain problems in such a way that the
proposed solution is KNOWN to not converge on an answer? An answer can
be perfectly findable IF you use a different representation, but there
are some ways of representing the problem that lead to a type of
solution that is completely incomputable.
This is an analogy: I suggest to you that the framework you have in
mind when you discuss the solution of the AGI problem is like those
broken representations.
-3---------experiential computers focus most learning, most models, and most
search on things that actually have happened in the past or on things that
in many ways are similar to what has happened in the past. This tends to
greatly reduce representational and search spaces.
When such a system synthesizes or perceives new patterns that have never
happened before the system will normally have to explore large search
spaces, but because of the capacity of brain level hardware it will have
considerable capability to do so. The type of hardware that will be
available for human-level agi in the next decade will probably have
sustainable cross sectional bandwidths of 10G to 1T messages/sec with 64Byte
payloads/msg. With branching tree activations and the fact that many
messages will be regional, the total amount of messaging could well be 100G
to 100T such msg/sec.
Lets assume our hardware has 10T msg/sec and that we want to read 10 words a
second. That would allow 1T msg/word. With a dumb spreading activation
rule that would allow you to: active the 30K most probably implications; and
for each of them the 3K most probable implications; and for each of them the
300 most probable implications; and for each of them the 30 most probable
implications. As dumb as this method of inferencing would be, it actually
would make a high percent of the appropriate multi-step inferences,
particularly when you consider that the probability of activation at the
successive stages would be guided by probabilities from other activations in
the current context.
Of course there are much more intelligent ways to guide activation that
this.
Also it is important to understand that at every level in many of the
searches or explorations in such a system there will be guidance and
limitations provided by similar models from past experience, greatly
reducing the amount of or the number of explorations that are required to
produce reasonable results.
-4---------Michael Collins a few years ago had was many AI researches
considered to be the best grammatical parser, which used the kernel trick to
effectively match parse trees in, I think it was, 500K dimensions. By use
of the Kernel trick the actual computation usually was performed in a small
subset of these dimensions and the parser was relatively efficient.
-5---------Hecht-Nielsen's sentence completion program (produced by his
"confabulation" see http://r.ucsd.edu), just by appropriately tying together
probabilistic implications learned from sequences of words, automatically
creates grammatically correct sentences that are related to a prior
sentense, allegedly without any knowledge of grammar, using millions of
probability activations per word, without any un-computable combinatorial
explosion. The search space that is being explored at any one time
theoretically is considering more possibilities than there are particles in
the known universe -- yet it works. At any given time several, lets, say 6
to 12 word or phrase slots can be under computation, in which each of
approximately 100K or so words or phrases is receiving scores. One could
consider the search space to include each of the possible words or phrase
being considered in each of those say 10 ordered slots as the possible
permutation of 10 slot fillers each chosen from a set of about 10^5 words or
phrases, a permuation that has (10^5)!/(10^4)! possibilities. This is a
very large search space -- just 100!/10! is over 10^151¸and (10^5)!/(10^4)!
is much, much, much larger space than that -- and yet it all compute with
somewhere within several orders of magnitude of a billion opps. This very
large search space is actually handled with a superposition of probabilities
(somewhat as in quantum computing) which are collapsed in a sequential
manner, in a rippling propagation of decisions and ensuing probability
propagations.
So Richard there are ways to do searches efficiently in very high
dimensional spaces, including in the case of confabulation spaces that are
in some ways trillions and trillions of times larger than the known universe
-- all on relatively small computers.
So lift thine eyes up unto Hecht-Nielsen -- (and his cat with whom he
generously shares credit for Confabulation) -- and believe!
These models you are talking about are trivial exercises in public
relations, designed to look really impressive, and filled with hype
designed to attract funding, which actually accomplish very little.
Please, Ed, don't do this to me. Please don't try to imply that I need
to open my mind any more. Th implication seems to be that I do not
understand the issues in enough depth, and need to do some more work to
understand you points. I can assure you this is not the case.
Richard Loosemore
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=8660244&id_secret=71681013-a60af6
