Johannes Zillmann wrote:
please correct me if i'm wrong, but if i understood
all right there are 2 choices...
(1) message based communication
(2) stream based communication
In case of (2) you won't come along without one thread
per connection.
In general, you are correct. But Nutch's IPC is somewhere between.
Requests and responses are discreet messages, but of potentially
unlimited size, and not length-prefixed. So streams and threads are
required to parse a request. But we don't expect individual clients to
be hammering the server with repeated requests, rather with periodic
requests every second or so. When a connection is between requests it
needs no thread. Thus a thread gets input on a connection until a
request is parsed, then disassociates itself from that connection.
Things only get complicated if multiple requests from a client are
merged into a single i/o event. This is unlikely, but when it occurs
that thread is forced to immediately handle another request from the
same client, which could result in the starving of other clients. But
as long as most individual clients don't make requests too quickly this
shouldn't be a problem.
But I still think I'm most likely to just use kernels and JVMs that
efficiently handle lots of threads and avoid the complexity of async nio.
Doug
