[
https://issues.apache.org/jira/browse/DERBY-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Matrigali updated DERBY-3732:
----------------------------------
With respect to the skip discussion. At least in the datatype case where the
stream is coming out of store as part of a select then derby "owns" the skip
code, and in that case there are array buffers sitting around which allow for
a very efficient skip. The code has multiple classes built on top of each
other but the interesting ones are:
OverflowInputStream.java (this is the key one - it is the type of stream store
always returns for any long type)
BufferedByteHolderInputStream.java
ByteHolder.java
ByteHolderInputStream.java
With more work we could do an even better job of this scan for length, but I
think it would take some interface
changes. The current code I think will read the disk bytes to the buffer
manager bytes, and then copy the
buffer manager bytes to a private array and then skip those bytes. Logically
no need to copy the bytes if we know
that we are just going to skip them - but would take some code and maybe some
interface change. Probably
a better thing to work on would be to just always insure that the length is at
the front of the blob. I am going to
file a JIRA with some ideas on this.
> SQL Length function materializes lob into memory
> ------------------------------------------------
>
> Key: DERBY-3732
> URL: https://issues.apache.org/jira/browse/DERBY-3732
> Project: Derby
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 10.3.3.0, 10.4.1.3, 10.5.0.0
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: derby-3732_diff.txt, derby-3732_proto_diff.txt,
> derby-3732_skip_diff.txt, LengthLargeLob.zip, LengthThruBlob.java
>
>
> Currently the SQL length function materializes the entire lob into memory. In
> SQLBinary.getLength() we have
> public final int getLength() throws StandardException
> {
> if (stream != null) {
> if (streamValueLength != -1)
> return streamValueLength;
> }
> return (getBytes() == null) ? 0 : getBytes().length;
> }
> Which actually is doubly bad because we call getBytes twice and materialize
> it twice.
> It would be good to read the length from the stream if available and
> otherwise stream the value to get the length, rather than materializing it
> into memory.
> To reproduce, run the attached repro.
> java -Xmx16M LengthLargeLob
> It gives an out of memory exception
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.derby.iapi.types.SQLBinary.readFromStream(SQLBinary.java:415)
> at
> org.apache.derby.iapi.types.SQLBinary.readExternal(SQLBinary.java:318)
> at org.apache.derby.iapi.types.SQLBinary.getValue(SQLBinary.java:220)
> at org.apache.derby.iapi.types.SQLBinary.getBytes(SQLBinary.java:210)
> at org.apache.derby.iapi.types.SQLBinary.getLength(SQLBinary.java:250)
> at
> org.apache.derby.impl.sql.execute.BaseActivation.getDB2Length(BaseActivation.java:1684)
> at
> org.apache.derby.exe.acf81e0010x011axa317x5db8x0000003d9dc81.e1(Unknown
> Source)
> at
> org.apache.derby.impl.services.reflect.DirectCall.invoke(ReflectGeneratedClass.java:141)
> at
> org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.doProjection(ProjectRestrictResultSet.java:497)
> at
> org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:291)
> at
> org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:460)
> at
> org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:423)
> ... 2 more
> [
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
