nfsantos commented on PR #1130: URL: https://github.com/apache/jackrabbit-oak/pull/1130#issuecomment-1748183653
> In this approach we are adding some processing to be done by mongo download thread which is the main bottleneck. Would it make sense to do these transformations on download task thread? The Mongo driver uses the thread that calls the Mongo iterator to perform the request to the server and to deserialize the response, so it is the mongo-dump thread that is doing all this work, there is no separate mongo download thread here. Before this PR, the download and transform threads would do this work, where the BSON stream is provided by Mongo to the codec: - mongo-dump - BSON stream -> BasicDBObject (OOTB codec) - transform thread - BasicDBObject -> NodeDocument -> NodeStateEntry -> serialize to buffers Now: - mongo-dump - BSON stream -> NodeDocument (custom codec) - transform thread - NodeDocument -> NodeStateEntry -> serialize to buffers And the transformation BSON stream -> NodeDocument takes the same or less time than to BasicDBOObject (edited) Overall, there is less work on the transform thread. The mongo-dump thread is doing the same work. I did not find a way to pull the BSON Stream -> X out of the mongo download thread, this is controlled by the mongo java drivers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
