[
https://issues.apache.org/jira/browse/DIRSERVER-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Norval Hope updated DIRSERVER-1161:
-----------------------------------
Attachment: streaming_log_output.txt
streaming_logging.patch
The .patch attached adds some extra logging to SearchHandler and ExecutorFilter
(in MINA) to make the problem apparent, and adds some extra settings in
apacheds/server-main/log4j.properties to ensure the new messages are logged.
I checked out
http://svn.apache.org/repos/asf/directory/apacheds/branches/1.0-with-dependencies
to a directory called "ad" and then checked out
http://svn.apache.org/repos/asf/mina/tags/1.0.3 to a directory called "mina"
under the top-level "ad", so the patch also includes building of this custom
MINA 1.0.3 in the top-level pom.xml.
Finally I've attached some log output for a search triggered from JXplorer for
all DNs under ou=system. Note how in attached .txt file we see "size of queue
is currently=17" without any writing being attempted until the final response
done is queued, at which time all results are written. I have verified when
running an LDAP client in the debugger (JMeter) that it sees no search results
until the done has been written to the session and the NIO structures are
flushed (and the synchronized block exited). I have also verified this lack of
streaming applies to searches returning 100k results, so it's not a simple
timing issue or anything simple like that.
> search results are not streamed to the client until final done response is
> queued
> ---------------------------------------------------------------------------------
>
> Key: DIRSERVER-1161
> URL: https://issues.apache.org/jira/browse/DIRSERVER-1161
> Project: Directory ApacheDS
> Issue Type: Bug
> Components: core
> Affects Versions: 1.0
> Environment: JDK 1.5.0_11
> Reporter: Norval Hope
> Attachments: streaming_log_output.txt, streaming_logging.patch
>
> Original Estimate: 120h
> Remaining Estimate: 120h
>
> Search results accumulate in Events on the SessionBuffer.eventQueue within
> ExecutorFilter.fireEvent() until final done response for the search is
> written to the session and then all results for the search (possibly millions
> depending on the search and state of the directory) are written out at once.
> This is a big problem for scalability and I gather from previous
> correspondence with Alex that this behaviour is unexpected.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
