Some follow-up :
I have done many tests, and I successfully got a working server without
having all the messages stored in memory, simply by changing the way the
ExecutorFilter is instanciated. In LdapService, line 361, instead of :
// Now inject an ExecutorFilter for the write operations
// We use the same number of thread than the number of IoProcessor
// (NOTE : this has to be double checked)
((DefaultIoFilterChainBuilder)chain).addLast( "executor",
new ExecutorFilter( new OrderedThreadPoolExecutor(
getTcpTransport().getNbThreads() ),
IoEventType.WRITE ) );
use :
// Now inject an ExecutorFilter for the write operations
// We use the same number of thread than the number of IoProcessor
// (NOTE : this has to be double checked)
((DefaultIoFilterChainBuilder)chain).addLast( "executor",
new ExecutorFilter( new OrderedThreadPoolExecutor(
getTcpTransport().getNbThreads() ) ) );
Now, the executor is used in both ways, and it seems to work much better.
I still have to understand how the ExecutorFilter() is working in MINA,
but at least, I think we cant release MINA 2.0.0 without a clear
documentation about the way to use this executor filter !
Andrea Gariboldi wrote:
Hi Martin,
Emmanuel found that there is a threading design problem within MINA:
The IoProcessor that is handling the search request and putting the encoded
results in the WriteQueue,
is the same that should dequeue those results from the queue and send it. So
once it has finished
with the request processing it checks for things to write and write all the
results in the queue, but
if you have long searches you will most likely get an OOM exception, because
of the growing queue.
Andrea
2009/4/3 Martin Alderson <[email protected]>
Hi Andrea,
Just to let you know, this looks to be the same problem that is being
discussed in my OutOfMemoryError thread on the users mailing list.
It still seems to be happening in the latest ApacheDS trunk. I don't
really understand why it is happening yet though.
Martin
Andrea Gariboldi wrote:
Hi guys,
i was testing the oracle partition and it looks like the mina filter
chain
buffers the search results until
the search ends and then starts message encoding end sending... Is this a
protocol constraint or a bug?
(org.apache.directory.server.ldap.handlers.SearchHandler : 326 snippet:
while ( (count < sizeLimit ) && cursor.next() )
{
if ( session.getIoSession().isClosing() )
{
break;
}
ClonedServerEntry entry = cursor.get();
session.getIoSession().write( generateResponse( session, req,
entry ) ); // here the message is "writed" trought mina filters, but is
not
encoded until the end of this loop...
count++;
}
)
Doing a long search (1000+ entries) results in long waits for the first
result on the client and java heap consumption.
am i missing something?
Andrea
--
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org
