Vladimir Nesov wrote:
Richard,
I'll try to summarize my solutions to these problems which allow to
use a network without need for explicit copying of instances (or any
other kind of explicit allocation of entities which are to correspond
to instances). (Although my model also requires ubiquitous induction
between nodes which disregards network structure.)
Basic structure of network: network is 'spiking' in the sense that it
operates in real time and links between nodes have a delay. Input
nodes send in the network sensory data, output nodes read actions. All
links between nodes can shift over time and experience through
induction. Initial configuration specifies simple pathways from input
to output, shifting of links changes these pathways, making them more
intricate to reflect experience.
Scene (as a graph which describes objects) is represented by active
nodes: node being active corresponds to feature being included in the
scene. Not all features present in the scene are active at the same
time, some of them can activate periodically, every several tacts or
more, and some other features can be represented by summarizing
simplified features (node 'apple' instead of 3D sketch of its
surface).
Network edges (links) activate the nodes. If condition (configuration
of nodes from which link originates) for a link is satisfied, and link
is active, it activates the target node.
Activation in the network follows a variation of Hebbian rule,
'induction rule' (which is essential for mechanism of instance
representation): link becomes active (starts to activate its target
node) only if it observed that node to be activated after condition
for link was satisfied in a majority of cases (like 90% or more). So,
if some node is activated in a network, there are good reasons for
that, no blind association-seeking.
Representation of instances. If scene contains multiple instances of
the same object (or pattern, say an apple), and these patterns are not
modified in it, there is no point in representing those instances
separately: all places at which instances are located ('instantiation
points', say places where apples lie or hang) refer to the same
pattern. The only problem is modification of instances in specific
instantiation points.
This scene can be implemented by creating links from instantiation
points to nodes that represent the pattern. As a result, during
activation cycle of represented scene, activation of instantiation
points leads to activation of patterns (as there's only one pattern
for each instantiation point, so induction rule works in this
direction), but not in other direction (as there are many
instantiation points for the pattern, none of them will be a target of
a link originating from the pattern).
This one-way activation results in a propagation of 'activation waves'
from instantiation points to the pattern, so that each wave 'outlines'
both pattern and instantiation point. These waves effectively
represent instances. If there's a modifier associated with specific
instantiation point, during an activation wave it will activate during
the same wave as pattern does, and as a result it can be applied to
it. As other instantiation points refer to the pattern 'by value',
pattern at those points won't change much.
Also, this way of representing instances is central to extraction of
similarities: if several objects are similar, they will share some of
their nodes and as a result their structures will influence one
another, creating a pressure to extract a common pattern.
I have questions at this point.
Your notion of "instantiation point" sounds like what I would call an
"instance node" which is created on the fly.
There is nothing wrong with this in principle, I believe, but it all
depends on the details of how these things are handled. For example, it
requires a *substantial* modification of the neural network idea to
allow for the rapid formation of instance nodes, and that modification
is so substantial that it would dominate the behavior of the system. I
don't know if you follow the colloquialism, but there is a sense in
which the tail is wagging the dog: the instance nodes are such an
important mechanism that everything depends on the details of how they
are handled.
So, to consider one or two of the details that you mention. You would
like there to be only a one-way connection between the generic node (do
you call this the "pattern" node?) and the instance node (instantiation
point?), so that the latter can contact the former, but not vice-versa.
Does this not contradict the data from psychology (if you care about
that)? For instance, we are able to see a field of patterns, of
different colors, and then when someone says the phrase "the green
patterns" we find that the set of green patterns "jumps out at us" from
the scene. It is as if we did indeed have links from the generic
concept [green pattern] to all the instances.
You may not care about the psychology, but should we disregard this fact
and assume we can get an AGI system to work without generic->instance links?
Another question: what do we do in a situation where we see a field of
grass, and think about the concept [grass blade]? Are there individual
instances for each grass blade? Are all of these linked to the generic
concept of [grass blade]?
I have no big problems with your model so far (if I understand it
correctly, which I am not sure I do) but it sits at the level of what I
would call the beginnings of a framework, rather than a complete model.
My goal is to build a complete framework, and I have been working on
this for many years. It is similar to the direction you are heading,
but goes further along its particular path.
Creation of new nodes. Each new node during a creation phase
corresponds to an existing node ('original node') in the network.
During this phase (which isn't long), each activated link that
connects to original node (both incoming and outgoing connections) is
copied so that in a copy original node is substituted by a new node.
As a result, new node will be active in situations in this original
node activated during creation of the new node. New node can represent
episodic memory or more specific subcategory of category represented
by original node. Initially, new node doesn't influence behavior of
the system (as it's activated in a subset of tacts in which original
node can activate), but because of this difference it can obtain
inductive links different from those that fit original node.
Again, this is broadly along the same lines that I think.
What is different is that I see many, many possible ways to get these
new-node creation mechanisms to work (and ditto for other mechanisms
like the instance nodes, etc.) and I feel it is extremely problematic to
focus on just one mechanism and say that THIS is the one I will
implement because .... "I think it feels like a good idea".
The reason I think this is a problem is that these mechanisms have
system-wide consequences (i.e. they give rise to global behaviors) that
are not necessarily obvious from the definition of the mechanism, so we
need to build a simulation to find out what those mechanisms *really* do
when they are put together and allowed to interact.
Rather than decide that I think one particular set of mechanisms is the
BEST one ahead of time, therefore, my strategy is to find ways to start
with a general framework, THEN create large numbers of
mechanism-candidates for each one of these issues (instance node format,
new node creation, etc etc), then explore the consequences of these
choices empirically. Doing large numbers of exploratory simulations
with different choices for the mechanisms.
I can show you a paper of mine in which I describe my framework in a
little more detail. If you have a paper in which you have described
yours in more detail I would be interested to read it.
Richard Loosemore
On Dec 5, 2007 4:47 AM, Richard Loosemore <[EMAIL PROTECTED]> wrote:
Dennis Gorelik wrote:
Richard,
3) A way to represent things - and in particular, uncertainty - without
getting buried up to the eyeballs in (e.g.) temporal logics that nobody
believes in.
Conceptually the way of representing things is described very well.
It's Neural Network -- set of nodes (concepts), when every node can be
connected with the set of other nodes. Every connection has it's own
weight.
Some nodes are connected with external devices.
For example, one node can be connected with one word in text
dictionary (that is an external device).
Do you see any problems with such architecture?
Many, unfortunately.
Too many to list all of them. A couple are: you need special extra
mechanisms to handle the difference between generic nodes and instance
nodes (in a basic neural net there is no distinction between these two,
so the system cannot represent even the most basic of situations), and
you need extra mechanisms to handle the dynamic creation/assignment of
new nodes, because new things are being experienced all the time.
These extra mechanisms are so important that is arguable that the
behavior of the system is dominated by *them*, not by the mere fact that
the design started out as a neural net.
Having said that, I believe in neural nets as a good conceptual starting
point.
It is just that you need to figure out all that machinery - and no one
has, so there is a "representation" problem in my previous list of problems.
Richard Loosemore
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?&;
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=8660244&id_secret=72289707-0fa645
