[
https://issues.apache.org/jira/browse/DERBY-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Matrigali updated DERBY-3732:
----------------------------------
>Looking at a possible quick fix approach as opposed to reworking the type code.
>If I insert the lob using length, I should have a streamValueLength available.
>When we call >getLength() stream is set but streamValueLength is not. Below is
>the trace when the stream is >set. It seems to me readRecordFromArray should
>be able to tell the length and pass that to >setStream(). Does that sound
>reasonable?
No I don't think this will work. store does not really know anything about the
stream, it basically
just knows that it is a "long" column which means that it was asked to store a
column longer than a page. And the current format for long column does not
include any sort of length encoding for the entire column. It splits the long
column into multiple pieces and links them and only maintains length of each
link. In the case of blob the type itself has the length encoding in the
column itself. The description of the encoding of this length is located in
SQLBinary.java.
What I think would be easiest is that the getlength() call in SQLBinary could
read the first few bytes and determine the length and then "reset()" the stream
so that the other calls would be
unaffected. Also depending on how the stream was stored there may not be a
length in
the stream so get length would have to read all the stream and then again do
the reset. The
question is whether this reset is going to mess anything else up. Doing it
this way would
only cause overhead if getlength is called, rather than always reading it.
SQLBinary.readBinaryLength() is the call that reads the length from inside the
column at the
front of the stream.
> SQL Length function materializes lob into memory
> ------------------------------------------------
>
> Key: DERBY-3732
> URL: https://issues.apache.org/jira/browse/DERBY-3732
> Project: Derby
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 10.3.3.0, 10.4.1.3, 10.5.0.0
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: LengthLargeLob.zip, LengthThruBlob.java
>
>
> Currently the SQL length function materializes the entire lob into memory. In
> SQLBinary.getLength() we have
> public final int getLength() throws StandardException
> {
> if (stream != null) {
> if (streamValueLength != -1)
> return streamValueLength;
> }
> return (getBytes() == null) ? 0 : getBytes().length;
> }
> Which actually is doubly bad because we call getBytes twice and materialize
> it twice.
> It would be good to read the length from the stream if available and
> otherwise stream the value to get the length, rather than materializing it
> into memory.
> To reproduce, run the attached repro.
> java -Xmx16M LengthLargeLob
> It gives an out of memory exception
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.derby.iapi.types.SQLBinary.readFromStream(SQLBinary.java:415)
> at
> org.apache.derby.iapi.types.SQLBinary.readExternal(SQLBinary.java:318)
> at org.apache.derby.iapi.types.SQLBinary.getValue(SQLBinary.java:220)
> at org.apache.derby.iapi.types.SQLBinary.getBytes(SQLBinary.java:210)
> at org.apache.derby.iapi.types.SQLBinary.getLength(SQLBinary.java:250)
> at
> org.apache.derby.impl.sql.execute.BaseActivation.getDB2Length(BaseActivation.java:1684)
> at
> org.apache.derby.exe.acf81e0010x011axa317x5db8x0000003d9dc81.e1(Unknown
> Source)
> at
> org.apache.derby.impl.services.reflect.DirectCall.invoke(ReflectGeneratedClass.java:141)
> at
> org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.doProjection(ProjectRestrictResultSet.java:497)
> at
> org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:291)
> at
> org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:460)
> at
> org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:423)
> ... 2 more
> [
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
