[ http://issues.apache.org/jira/browse/DERBY-326?page=comments#action_12372448 ]
Tomohito Nakayama commented on DERBY-326: ----------------------------------------- I have surveyed around the drop in performance of streaming blob after the patch. After struggling to interpret the profiled informations, I found that java.net.SocketOutputStream.writemethod seems to takes much more part in processing than before the patch. I took this as *sign* of problem at client side and realized that client side handles buffer without information for length of information to be cached after the patch and that it would be reason of drop in performance. Then I tested with modified client driver that have buffer of 5 * 1024 * 1024 bytes, corresponding size of the streamed file. //Where I specified the size of buffer was next two part. //* org.apache.derby.client.am.Connection.commBufferSize_ and //* parameter for constructor of ByteArrayOutputStream called from org.apache.derby.client.net.Reply.getData when ddmScalarLen_ is -1. Result was as next. Avg time taken to read nullrows+ (ignore first run )=760ms This is faster than result before the patch. ==> ./before/blob/volume/result.txt <== Avg time taken to read nullrows+ (ignore first run )=806ms This result suggests that problem around buffer at client is the reason why the patch does not improve streaming blob. Now I think it is the reason of drop in performance of streaming blob that buffer at client side is handled without information of length to be buffered after the patch. > Improve streaming of large objects for network server and client > ---------------------------------------------------------------- > > Key: DERBY-326 > URL: http://issues.apache.org/jira/browse/DERBY-326 > Project: Derby > Type: Improvement > Components: Network Client, Performance, Network Server > Reporter: Kathey Marsden > Assignee: Tomohito Nakayama > Attachments: ClobTest.zip, DERBY-326.patch, DERBY-326_2.patch, > DERBY-326_3.patch, DERBY-326_4.patch, DERBY-326_5.patch, > DERBY-326_5_indented.patch, DERBY-326_6.patch, > ReEncodedInputStream.java.modifiedForLongRun > > Currently the stream writing methods in network server and client require a > length parameter. This means that we have to get the length of the stream > before sending it. For example in network server in EXTDTAInputStream we have > to use getString and getbytes() instead of getCharacterStream and > getBinaryStream so that we can get the length. > SQLAM Level 7 provides for the enhanced LOB processing to allow streaming > without indicating the length, so, the writeScalarStream methods in > network server DDMWriter.java and network client Request.java can be changed > to not require a length. > Code inspection of these methods seems to indicate that while the length is > never written it is used heavily in generating the DSS. One strange thing is > that it appears on error, the stream is padded out to full length with zeros, > but an actual exception is never sent. Basically I think perhaps these > methods need to be rewritten from scratch based on the spec requirements for > lobs. > After the writeScalarStream methods have been changed, then EXTDAInputStream > can be changed to properly stream LOBS. See TODO tags in this file for more > info. I am guessing similar optimizations available in the client as well, > but am not sure where that code is. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
