[
https://issues.apache.org/jira/browse/DERBY-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12606869#action_12606869
]
Kathey Marsden commented on DERBY-3732:
---------------------------------------
Looking at a possible quick fix approach as opposed to reworking the type code.
If I insert the lob using length, I should have a streamValueLength available.
When we call getLength() stream is set but streamValueLength is not. Below is
the trace when the stream is set. It seems to me readRecordFromArray should
be able to tell the length and pass that to setStream(). Does that sound
reasonable?
SQLBlob(SQLBinary).setStream(InputStream) line: 563
StoredPage.readRecordFromArray(Object[], int, int[], int[],
ArrayInputStream, StoredRecordHeader, RecordHandle) line: 5592
StoredPage.restoreRecordFromSlot(int, Object[], FetchDescriptor,
RecordHandle, StoredRecordHeader, boolean) line: 1497
StoredPage(BasePage).fetchFromSlot(RecordHandle, int, Object[],
FetchDescriptor, boolean) line: 459
HeapScan(GenericScanController).fetchRows(DataValueDescriptor[][],
RowLocation[], BackingStoreHashtable, long, int[]) line: 759
HeapScan.fetchNextGroup(DataValueDescriptor[][], RowLocation[]) line:
324
BulkTableScanResultSet.reloadArray() line: 327
BulkTableScanResultSet.getNextRowCore() line: 282
ProjectRestrictResultSet.getNextRowCore() line: 255
ProjectRestrictResultSet(BasicNoPutResultSetImpl).getNextRow() line:
460
EmbedResultSet40(EmbedResultSet).movePosition(int, int, String) line:
423
EmbedResultSet40(EmbedResultSet).next() line: 367
LengthLargeLob.main(String[]) line: 21
> SQL Length function materializes lob into memory
> ------------------------------------------------
>
> Key: DERBY-3732
> URL: https://issues.apache.org/jira/browse/DERBY-3732
> Project: Derby
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 10.3.3.0, 10.4.1.3, 10.5.0.0
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: LengthLargeLob.zip, LengthThruBlob.java
>
>
> Currently the SQL length function materializes the entire lob into memory. In
> SQLBinary.getLength() we have
> public final int getLength() throws StandardException
> {
> if (stream != null) {
> if (streamValueLength != -1)
> return streamValueLength;
> }
> return (getBytes() == null) ? 0 : getBytes().length;
> }
> Which actually is doubly bad because we call getBytes twice and materialize
> it twice.
> It would be good to read the length from the stream if available and
> otherwise stream the value to get the length, rather than materializing it
> into memory.
> To reproduce, run the attached repro.
> java -Xmx16M LengthLargeLob
> It gives an out of memory exception
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.derby.iapi.types.SQLBinary.readFromStream(SQLBinary.java:415)
> at
> org.apache.derby.iapi.types.SQLBinary.readExternal(SQLBinary.java:318)
> at org.apache.derby.iapi.types.SQLBinary.getValue(SQLBinary.java:220)
> at org.apache.derby.iapi.types.SQLBinary.getBytes(SQLBinary.java:210)
> at org.apache.derby.iapi.types.SQLBinary.getLength(SQLBinary.java:250)
> at
> org.apache.derby.impl.sql.execute.BaseActivation.getDB2Length(BaseActivation.java:1684)
> at
> org.apache.derby.exe.acf81e0010x011axa317x5db8x0000003d9dc81.e1(Unknown
> Source)
> at
> org.apache.derby.impl.services.reflect.DirectCall.invoke(ReflectGeneratedClass.java:141)
> at
> org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.doProjection(ProjectRestrictResultSet.java:497)
> at
> org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:291)
> at
> org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:460)
> at
> org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:423)
> ... 2 more
> [
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
