On Monday 06 October 2003 09:58 pm, Ian Clarke wrote:
> Tom Kaitchuck wrote:
> > When exactly did the massive network failure happen?
>
> It may already be fixed.
>
> > I've read all the NGrouting Docs, and I always had quiet reservations. I
> > never thought the 10 points model could work well enough to provide
> > internode routing. I just assumed that I would be pr oven wrong when it
> > all came together. However since this is not currently happening, I'll
> > just throw some thoughts out there.
>
> The problem is much more likely to be implementation bugs. Some serious
> bugs were found and fixed within the last three hours.
>
> > NGrouting uses 10 points to estimate the the specialization of each node
> > you contact to. My question, is in essence is, "how can this work?". This
> > model is all well and good if you are connecting to an existing network
> > as a client. You get a very good idea of where to make requests and
> > generally how to route your data. So when nodes route their first hop,
> > However what happens during the last few hops during an insert? Ten
> > points spread out over 0-2^160 is not many points.
>
> The points move, so they can concentrate in one area to represent
> detailed information about that part of keyspace. This is *much* more
> sophisticated than the existing mechanism which worked surprisingly well.
Yes, I got that. However in order to represent an aria of fast responce, you
need to be using at least three of those points. In reality perhaps four or
more. If that is the case, NGrouting is not well suited for multiple or
intermittent arias of specialization. That is fine, but it means that it
probably would not mix all that well with the existing network, because it is
hard for it to estimate the specialization of nodes that are not using it.
> > Also many nodes have a large number of arias of
> > specialization. (More than 10). Can nodes relay distinguish the
> > difference in specialization of two nodes which specialize in the same
> > (1/256^10)th of the keyspace? (That assumes 256 outgoing connections, and
> > 10 hops in). If so how, and does this work in the same way that the old
> > routing system did?
>
> Yes, because the mechanism we use is "scale free". See the
> documentation of NGR for an explanation of this.
Part of the larger issue that I was getting at later in my message, was that
NGrouting does not develop specialization as quickly as the old algorithm. It
also, in a poorly preforming network, IE due to bugs, it may degenerate and
not work at all.
One thing that could be done to combat this would be to force specialization.
Not in the since that a node is forced to a particular aria of
specialization, but just that is forced to develop one. (regardless of what
it is)
The best way to do this would be to have a single function that decided what
to do with data;
returnValue WhatDoIDoWithIncommingRequest( key, HTL, RequestingNode,
FailedRequestsTable, NodeTable, GlobalEstimates, EnviroData,
DataCurrentlyInStore)
{
...
return ReturnValue;
}
Where return value indicates, whether you:
Process the request or not.
Where you route the request.
Whether you will cache the data upon return.
What if any data should be removed from the store to make room for it.
Whether you should redirect yourself as the source.
Etc.
Of course I don't really think all this should be lumped into a single
function. The point is that ultimately all of those operations
(Request/Drop,Store/Remove,Where to route,Redirect to yourself/not,Open a new
connection/close an old one) all have the same goal. That is to give the best
overall throughput. The way this is done is to have each of them make their
decision using a non-alchemistic means of determining what is best given our
best prediction for what feature traffic will/should look like, using the
data available from the current and previous data processed. Note I say
should because there will always be a certain number of new nodes asking you
for data that is more or less random, because they have just connected to you
or they are the originator of the request. One should not let these data
requests dilute ones ability to specialize, because these requests are not
good at predicting future requests. But this must be done in a mannor besides
simply trying to exclude them from the dataset the algorithm makes it's
decisions on, and at the same time is non-alchemistic.
_______________________________________________
Devl mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
