[
https://issues.apache.org/jira/browse/DERBY-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Matrigali updated DERBY-3732:
----------------------------------
The 2 cases I try to think about with stream/blobs/clobs are always the
following:
1) The data is coming off the disk, through store as a stream, and back to user
somehow
- ie. select
2) The data is coming from user to be put into store as a stream - ie. insert.
Mostly the datatype code should not really care which of these 2 cases is in
effect, but
I think Resetable is a case that matters.
The case you are testing is #1, with getLength() being called by the query
itself on a stream
created by the store. That stream is always going to be a OverflowInputStream
when it comes
out of store - does not matter what type. But type code should not use that.
I think using
instanceof Resetable is the right thing.
In case 2 embedded I think the stream may come directly from user - but I am
not sure what
jdbc layer may do to it before it gets made into a SQLBinary. A different case
might be what
comes to the embedded engine after a stream gets sent in from a network client
across drda
and then into the embedded engine. In both of these cases I don't know if you
can actually
get getLength() called, but I also don't think you can count on resetting the
stream so seems
cleaner to leave the code as is for this case rather than have the code pretend
it could handle it. I don't really know a lot about case 2, so may be off
track. The fact that the stream has
the length encoding in it means that some derby code has already got involved
and modified
the stream the user provided.
I don't know if there are any other situations with streams available in the
datatype -
maybe triggers?
you probably should ignore the skip comment, better to get it right than worry
about optimizing it. Your fix should be way faster in case where length exists
at front, and be way less memory intensive in the other case so a good
incremental improvement.
> SQL Length function materializes lob into memory
> ------------------------------------------------
>
> Key: DERBY-3732
> URL: https://issues.apache.org/jira/browse/DERBY-3732
> Project: Derby
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 10.3.3.0, 10.4.1.3, 10.5.0.0
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: derby-3732_proto_diff.txt, LengthLargeLob.zip,
> LengthThruBlob.java
>
>
> Currently the SQL length function materializes the entire lob into memory. In
> SQLBinary.getLength() we have
> public final int getLength() throws StandardException
> {
> if (stream != null) {
> if (streamValueLength != -1)
> return streamValueLength;
> }
> return (getBytes() == null) ? 0 : getBytes().length;
> }
> Which actually is doubly bad because we call getBytes twice and materialize
> it twice.
> It would be good to read the length from the stream if available and
> otherwise stream the value to get the length, rather than materializing it
> into memory.
> To reproduce, run the attached repro.
> java -Xmx16M LengthLargeLob
> It gives an out of memory exception
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.derby.iapi.types.SQLBinary.readFromStream(SQLBinary.java:415)
> at
> org.apache.derby.iapi.types.SQLBinary.readExternal(SQLBinary.java:318)
> at org.apache.derby.iapi.types.SQLBinary.getValue(SQLBinary.java:220)
> at org.apache.derby.iapi.types.SQLBinary.getBytes(SQLBinary.java:210)
> at org.apache.derby.iapi.types.SQLBinary.getLength(SQLBinary.java:250)
> at
> org.apache.derby.impl.sql.execute.BaseActivation.getDB2Length(BaseActivation.java:1684)
> at
> org.apache.derby.exe.acf81e0010x011axa317x5db8x0000003d9dc81.e1(Unknown
> Source)
> at
> org.apache.derby.impl.services.reflect.DirectCall.invoke(ReflectGeneratedClass.java:141)
> at
> org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.doProjection(ProjectRestrictResultSet.java:497)
> at
> org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:291)
> at
> org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:460)
> at
> org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:423)
> ... 2 more
> [
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
