[
https://issues.apache.org/jira/browse/DERBY-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12612455#action_12612455
]
Knut Anders Hatlen commented on DERBY-3769:
-------------------------------------------
I think it is correct to fix this on the server side only. If we want the fix
on the client side, we need to give the client knowledge about maximum sizes
for different server versions (not a big issue right now, since the max
varchar/varbinary size hasn't changed) which sounds like unnecessary complexity
if it only fixes performance issues with the combination of old server and new
client.
I think the fix looks good. I checked the code in CallableLocatorProcedures,
and it seems to correctly request more data if the string is truncated, so the
fix should be fine. You may consider the following cosmetic changes:
1) Instead of using Limits.DB2_VARCHAR_MAXWIDTH, we could have a dedicated
constant (CLOBGETSUBSTRING_RETURN_LENGTH, or something) that we used both when
we declared the stored procedures (in DataDictionaryImpl) and to truncate the
return value in the procedures. This way it is easier to keep them consistent
if we at some point change the definition of the procedures.
2) The javadoc comments have paragraphs marked as implementation notes which
say that the length may be shorter than specified. I don't think these aren't
actually implementation notes, but crucial points of the API that anyone
programming against these procedures should be aware of.
3) The comments say that the length may be shorter because of client/server
communication. I believe it is more correct to say that the length may be
shorter than specified because the stored procedure returns a VARCHAR (or
VARBINARY) and can therefore not return strings whose length exceed the max
length for the data type.
4) Typo in javadoc for BLOBGETBYTES: smallar -> smaller
> Make LOBStoredProcedure on the server side smarter about the read buffer size
> -----------------------------------------------------------------------------
>
> Key: DERBY-3769
> URL: https://issues.apache.org/jira/browse/DERBY-3769
> Project: Derby
> Issue Type: Improvement
> Components: Network Server
> Affects Versions: 10.3.3.0, 10.4.1.3, 10.5.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Attachments: derby-3769-1a-buffer_size_adjustment.diff
>
>
> Derby has a max length for VARBINARY and VARCHAR, which is 32'672 bytes or
> characters (see Limits.DB2_VARCHAR_MAXWIDTH).
> When working with LOBs represented by locators, using a read buffer larger
> than the max value causes the server to process far more data than necessary.
> Say the read buffer is 33'000 bytes, and these bytes are requested by the
> client. This requests ends up in LOBStoredProcedure.BLOBGETBYTES.
> Assume the stream position is 64'000, and this is where we want to read from.
> The following happens:
> a) BLOBGETBYTES instructs EmbedBlob to read 33'000 bytes, advancing the
> stream position to 97'000.
> b) Derby fetches/receives the 33'000 bytes, but can only send 32'672. The
> rest of the data (328 bytes) is discarded.
> c) The client receives the 32'672 bytes, recalculates the position and
> length arguments and sends another request.
> d) BLOBGETBYTES(locator, 96672, 328) is executed. EmbedBlob detects that the
> stream position has advanced too far, so it resets the stream to position
> zero and skips/reads until position 96'672 has been reached.
> e) The remaining 328 bytes are sent to the client.
> This issue deals with points b) and d), by avoiding the need to reset the
> stream.
> Points a) and e) are also problematic if a large number of bytes are going to
> be read, say hundreds of megabytes, but that's another issue.
> It is unfortunate that using 32 K (32 * 1024) as the buffer size is almost
> the worst case; 32'768 - 32'672 = 96 bytes.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
