I've been looking into why node throughput is so low.
Here are a few things I've come up with:
0) Bad handling of complete routing failure in TreeRoutingTable.
I think TreeRoutingTable.RouteWalker.step() is trying too hard in
the case where it falls back on skipped node refs. Successive calls
can hammer on the same broken or almost broken noderefs over and over.
I supect that this can look almost like a DOS attack to the target
nodes.
I just checked in changes that give failed nodes an interval to
"rest" before they are retried again. I think that cooperatively backing
off in this manner will reduce network congestion. Request will be better
routed to nodes that can answer them. If the node really is
completely out of usable refs requests will fail quickly with RNFs instead of
waiting for all of the known bad refs to be contacted and fail.
1) It looks like the thread pools job queue is way too big.
Snippet from Freenet.node.Main.startNode:
if (node.maximumThreads > 0) {
tm = new ThreadPool(tg, 5, node.maximumThreads >> 1,
node.maximumThreads);
tm.start();
}
First, only half as many threads as are specified in
freenet.conf are actually created. This is somewhat
counter-intuitive to the end user.
Second, you get as many allowed queued jobs as threads.
This just seems wrong. I have instrumented the code and watched
the job queue fill up completely. If a message has 60 jobs ahead of
it, chances are good that it will be "stale" by the time it is run.
It's easy to reduce the size of the job queue. Figuring out what
to do to recover somewhat gracefully when it overflows is much more difficult.
I don't really know the answer.
I will not comment on the practice of using shift operators instead
of division...
2) Tuning the ratio of open connections and allowed threads.
I think the default maxNodeConnections (30) is too big for the default
value of maximumThreads (120 which is only 60 real threads). When the
network gets congested, message chains restart more often which
seems to eat up more threads.
I have been running my node with 10 connections and 240 threads (120 real
threads).
Thoughts?
-- gj
--
Freesites
(0.3) freenet:MSK at SSK@enI8YFo3gj8UVh-Au0HpKMftf6QQAgE/homepage//
(0.4) freenet:SSK at npfV5XQijFkF6sXZvuO0o~kG4wEPAgM/homepage//
_______________________________________________
Devl mailing list
Devl at freenetproject.org
http://lists.freenetproject.org/mailman/listinfo/devl
