[
https://issues.apache.org/jira/browse/DIRSERVER-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12588158#action_12588158
]
Emmanuel Lecharny commented on DIRSERVER-1161:
----------------------------------------------
Hi Norval,
I read all this thread in the train while coming back from Amsterdam, and what
I saw (and what I went through while debugging the server) was quite scary ...
I'm not sure that MINA in the current version still hold the data in a queue
before sending them, but anyway, if it's not the case, the slow JNDI client is
already a perfect example of what we should protect the server from !
Ok, after some minimal investigation, I found that MINA was calling the encoder
in the ProtocoalEncoderFilter and flush the resulting bytes directly to the
client, but then try to send the message again, which has obviously already
been sent. Here is the code :
ProtocolCodecFilter.filterWrite() {
...
ProtocolEncoder encoder = getEncoder(session);
ProtocolEncoderOutputImpl encoderOut = getEncoderOut(session,
nextFilter, writeRequest);
try {
encoder.encode(session, message, encoderOut);
// Here, the encoded message is sent.
encoderOut.flush();
// Here an empty message is sent...
nextFilter.filterWrite(session, new WriteRequest(
new MessageByteBuffer(writeRequest.getMessage()),
writeRequest.getFuture(), writeRequest.getDestination()));
I don't know if this last empty message sent is related to the issue you
mentionned in your previous comment, as I haven't yet thoroughly tested the
code, but it might be a problem.
I think we might went to wait until the data are _really_ read by the client
just after the encoderOut.flush() call before sending the next result, and this
would protect the server from the issue you mentioned before.
Side remark : MINA migth have a cool API, but when it comes to debug it and
getting into the guts of it, this is a nightmare. I just think that many part
of it is just over-complicated, defeating the moto : "Write code for the
Average Programmer"
> search results are not streamed to the client until final done response is
> queued
> ---------------------------------------------------------------------------------
>
> Key: DIRSERVER-1161
> URL: https://issues.apache.org/jira/browse/DIRSERVER-1161
> Project: Directory ApacheDS
> Issue Type: Bug
> Components: core
> Affects Versions: 1.0
> Environment: JDK 1.5.0_11
> Reporter: Norval Hope
> Attachments: apacheds_1.5.1_streaming.patch,
> apacheds_1.5.1_streaming_log_output.txt, installers_1.5.1_streaming.patch,
> mina_1.1.2_streaming.patch, pom.xml, streaming_log_output.txt,
> streaming_logging.patch
>
> Original Estimate: 120h
> Remaining Estimate: 120h
>
> Search results accumulate in Events on the SessionBuffer.eventQueue within
> ExecutorFilter.fireEvent() until final done response for the search is
> written to the session and then all results for the search (possibly millions
> depending on the search and state of the directory) are written out at once.
> This is a big problem for scalability and I gather from previous
> correspondence with Alex that this behaviour is unexpected.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
