Hi,
we were dscussing about the messageSent event in the recent past,
wondering if this event was useful or not.
Today, while trying to deal with a problem on an application that gets
some OOM when sending big messages, we started to discuss with Julien
about the best way to solve the issue.
Basically, what happens is that an application received a request, then
build a big response, and tries to send this big response in small
chunks, expecting that all the chunks will be sent one by one, instead
of being stored in a temporary queue waiting for the client socket to be
ready. Thet does not work : everything is stored first in memory, then sent.
The reason is that the writes are processed after the reads, so if you
try to write while processing the read, the written data will be stored
in a queue (so in memory).
So how to deal with this problem ? In MINA 2, there are two ways :
- you keep doing this, but you add an executor in the filter chain, in
order to have a new thread that can process the writes
- you wait for the chunk of data to be written into the socket before
trying to write some new data
The first solution will still stress a lot our server, because writing
data in a queue will be faster than writing the data in the socket.
The second solution requires that you wait for the messageSent event to
send more data. In this case, your cod will look like :
messageReceived() {
data = getAChunkOfData()
session.write( data )
}
messageSent() {
data = getAChunkOfData()
if ( data != null ) {
session.write( data )
}
}
For every chunk of data completely pushed into the soccket, you will get
a messageSent event generated. You have the guarantee that the chunks of
data won't stay in memory.
So the important point here is that the messageSent event is mandatory,
and we should most certainly use it in MINA 3.
Now, ther eis one more tricky part we need to rething : the WriteFuture
usage.
Let's say one wants to do something like :
messageReceived() {
while ( ( data = getAChunkOfData() ) != null ) {
writeFuture = session.write( data )
writeFuture.get()
}
}
then you have not solved your memory issue, as the writeFuture will be
signaled only when the chuck of data will have been added in the queue.
If we modify the writeFuture to be signaled when the data are really
written in the socket, then with MINA 2, you will be blocked forever, as
the thread processing the writes is the same thred you are using to
process the reads (and you are already using it).
That would be different if we were using a different thread for the
selector and for the processor, something we should probably do in MINA
3 anyway...
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
