-----Original Message----- From: Emmanuel Lécharny [mailto:[email protected]] Sent: Monday, February 14, 2011 2:36 AM To: [email protected] Subject: Re: where to place the ExecutorFilter?
On 2/13/11 3:23 PM, Guy Itzhaki wrote: > Emmanuel, thanks for the response. > > 1. The server processes incoming messages using the protocolCodecFilter. > Currently it is the protocolCodecFilter responsibility to read the ENTIRE > message and save it in the hard disk for future processing by the Handler. > Reading few hundred MBs is an expensive process and might take a while, so > dropping the first ExecutorFilter means that the NIOProcessor thread will > handle this (long) task. If I'm not mistaken this is a bad choice because I > have only few Processors so I'll easily cause my server to not > excepting\processing other requests. Am I right? If I'm, maybe a better > design for my server would be to move the (expensive) file reading to the > handler... I think that it's probably better to move the processing of the huge incoming messages into the handler, yes. Now, considering that writing data on disk is 1000 times more expensive that processing them in memory (pulling this number out of my hat, but let's use this number for the sake of the demonstration), then your bottleneck will be the disk, not the threads : no matter how many threads you will use, you will still be bounded by how fast is your disk... I'll move the reading of the file to the handler as you suggested. Note that my message composed of two: a meta data header and the file itself. I'll read the meta data in the ProtocolCodec and leave the file reading for the handler. My question then is how can I read the rest of the message from the handler? The messageReceived method has 2 params one is the session, I didn't find any example of reading directly from the session? > 2. In order to be efficient and consume as less memory as it can, the server > sends the big files using FileWithHeaderWriteFilter This class is not a MINA class, AFAICT > (which extends AbstractStreamWriteFilter) I've noted that even if I have an > ExecutorFilter in the chain Mina does not use its threads, instead it uses > NIOProcessor threads. Ultimately, when the message has transited through the chain, it's queued into an IoProcessor, so adding an executor expecting that more threads will be used to send the data to the client is simply useless. Keep in mind that once the socket is opened, it's associated with a selector, and what is important is the thread having access to this selector. Sharing this selector through many threads would not help... > (Again,) this is an issue as I would expect my server to process > simultaneously large number of files and not only few. How can I force Mina > to use other threads than the NIOProcessor threads for that? Increase the number of IoProcessors. Again, at some point, writing data into sockets is fast, compared to the time it takes to send them on the physical network, so I'm not sure that having tens of IoProcessor helps a lot. An IoProcessor just read the queue, and inject the data into the socket, if it's ready for write. It does that repetitively, as soon as he select() method signal that at least one socket is ready. This is definitively not the bottleneck. If you need to increase the throughput, then you have to think about having more than one server... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com Click here to report this message as SPAM: http://vsp.ateranetworks.com/ReportSpam.php?sid=a955af5a804820ad90c87dc795acb80f_9e45831b18e75d23118e18ddb0b4646d -- Powered by ATERA Networks --
