On 2/13/11 3:23 PM, Guy Itzhaki wrote:
Emmanuel, thanks for the response.
1. The server processes incoming messages using the protocolCodecFilter.
Currently it is the protocolCodecFilter responsibility to read the ENTIRE
message and save it in the hard disk for future processing by the Handler.
Reading few hundred MBs is an expensive process and might take a while, so
dropping the first ExecutorFilter means that the NIOProcessor thread will
handle this (long) task. If I'm not mistaken this is a bad choice because I
have only few Processors so I'll easily cause my server to not
excepting\processing other requests. Am I right? If I'm, maybe a better design
for my server would be to move the (expensive) file reading to the handler...
I think that it's probably better to move the processing of the huge
incoming messages into the handler, yes. Now, considering that writing
data on disk is 1000 times more expensive that processing them in memory
(pulling this number out of my hat, but let's use this number for the
sake of the demonstration), then your bottleneck will be the disk, not
the threads : no matter how many threads you will use, you will still be
bounded by how fast is your disk...
2. In order to be efficient and consume as less memory as it can, the server
sends the big files using FileWithHeaderWriteFilter
This class is not a MINA class, AFAICT
(which extends AbstractStreamWriteFilter) I've noted that even if I have an
ExecutorFilter in the chain Mina does not use its threads, instead it uses
NIOProcessor threads.
Ultimately, when the message has transited through the chain, it's
queued into an IoProcessor, so adding an executor expecting that more
threads will be used to send the data to the client is simply useless.
Keep in mind that once the socket is opened, it's associated with a
selector, and what is important is the thread having access to this
selector. Sharing this selector through many threads would not help...
(Again,) this is an issue as I would expect my server to process simultaneously
large number of files and not only few. How can I force Mina to use other
threads than the NIOProcessor threads for that?
Increase the number of IoProcessors. Again, at some point, writing data
into sockets is fast, compared to the time it takes to send them on the
physical network, so I'm not sure that having tens of IoProcessor helps
a lot. An IoProcessor just read the queue, and inject the data into the
socket, if it's ready for write. It does that repetitively, as soon as
he select() method signal that at least one socket is ready. This is
definitively not the bottleneck.
If you need to increase the throughput, then you have to think about
having more than one server...
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
