On Mon, Nov 04, 2002 at 01:20:27PM -0000, Dave Hooper wrote:
> > Please do not use
> > the 0.6 as the overload threshold - with 80 threads that leaves only 12
> > for the node to process requests and new connections, which is far too
> > little.
>
> Bit of a dumbass question - what do all the threads actually *do*? As in,
> what are the specific tasks that a thread does? Are all threads created
> equal or are some created to do something specific - once a thread has been
> removed from the idle pool does it 'do one thing' before being returned to
> the idle pool or can it change role without being idled and unidled?
If you want to know what your threads are doing, you can check on in the
environment page. Scroll down and there is a list of thread consumers.
As I write this, my node (which has become very busy, probably because
it is one of the only nodes not rejecting everything) is using 92
threads. The distribution is:
60 Open connections to other nodes (the max is 0.4 *
maximumThreads and I have that set 150.)
14 Data transfers. These are threads sending data from the
datastore in response to requests.
11 Connections that are in the process of being opened and
closed (mostly opened.)
6 Messages that are actively being handled (4 DataRequests,
1 QueryRestarts, and 1 QueryRejected.)
1 Local connection (this is the thread that is serving the page
I loaded containing the information.)
When a pooled thread is finished with a task, it returns to the pool and
can be used again. Being "idled and unidled" is not a heavy operation at
all.
> If the only reason for having so many threads is to prevent IO blocking
> then we really ought to use asynchronous IO. Since I don't know Java I
> don't know how realistic this is but obviously it must be an issue because
> other people have been talking about it and apparently we still aren't
> using it. My guess is, assuming the JVM supports asynchronous IO calls at
> all, that the IO scheduling is handled almost entirely by the OS not the
> JVM, whereas synchronous IO calls not only require more OS resources but
> also more JVM resources too, simply because of the number of threads
> required.
Java had no official support for asynchronous IO at all previous to
version 1.4. We are still targeting 1.1. There is still no free JVM
solution that can offer it to us, although the package in question has
been implemented as part of the GNU Classpath libraries (which as far as
I can tell are a collection of great code being used for nothing.)
> If fred could use asynchronous IO, assuming JVM support, how many threads
> would be necessary? Like, three or four rather than a hundred?
That depends how you write it, of course. But Fred has been written
around the old java IO system from the ground up (because that is all we
had to work with), so trying to move to asynch IO would imply a huge
change. Making it use asynch IO and few threads would imply rewriting
all the connection handling, the data transfering, and the establishing
of connections (of which the first two are seriously non trivial). Then
all the message handling code would need to be reworked, to cooperate
with the different behavior. I don't see this as realistic - now, or any
time.
What might be possible is making connections use asynch IO but leaving
the other code as it is. That would drop 72 of the 92 threads used
above, but continue to use a comparatively large number.
> Or are all the additional threads used just to service the computational
> aspects of request and response, such as encryption, fec, etc?
Everything that can use a thread does, basically.
--
Oskar Sandberg
oskar at freenetproject.org
_______________________________________________
devl mailing list
devl at freenetproject.org
http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/devl
