--- Begin Message ---
Thanks, that seems reasonable. I have reviewed the rest and it
looks good. Once tests pass on my machine I will commit.
Sunitha Kambhampati wrote:
> Mike Matrigali wrote:
>
>> I am looking at commiting this, one question:
>> o in ReaderToUTF8Stream.java you changed close() to not do a
>> reader.close() - why is this ok?
>>
>>
>>
> Thanks Mike for reviewing this patch.
>
> E.g.: So user application calls the following api to set the stream.
> ps.setCharacterStream(i, mystream, pos) //mystream is user 's stream.
>
> Internally, ReaderToUTF8Stream is the stream that wraps the user's
> stream as LimitReader object. Thus calling reader.close() will close
> the user's stream and I think that is incorrect. I think it is the
> responsibility of the application to either close the stream or do
> whatever the application wishess and derby should not close the 'user's
> stream' internally.
>
> E.g. The user could have passed in a Resetable stream and then after
> the statement execution can reset his stream and use the same object to
> do some other work. If derby closes the user's stream internally, any
> other call on the stream would throw an exception and user is forced to
> create another stream object. .
>
> Also previously (before this fix), ReaderToUTF8Stream.close() was never
> exercised,but now I believe it correctly does return resources back to
> the VM.
>
> Sunitha.
>
>> Sunitha Kambhampati (JIRA) wrote:
>>
>>
>>
>>> [ http://issues.apache.org/jira/browse/DERBY-500?page=all ]
>>>
>>> Sunitha Kambhampati updated DERBY-500:
>>> --------------------------------------
>>>
>>> Attachment: Derby500.diff.txt
>>>
>>> Background :
>>> In Derby, when a stream is set as a parameter value, the wrapper
>>> stream object used for character data is ReaderToUTF8Stream and for
>>> binary data it is RawToBinaryFormatStream.Both these stream objects
>>> on read() return data in a format that is used to store the
>>> respective datatype value. E.g in case of char, the characters read
>>> from the user stream are converted using utf-8 derby specific
>>> encoding and read calls return the data as expected by store layer.
>>> Beginning 2 bytes either have the utflen or has zeroes, or if it is a
>>> long string, then the value is ended with the special marker 0xE0 ,
>>> 0x00, 0x00. For binary data, the stream data is prepended with 4 zeroes.
>>> Problem:
>>> once,the stream has been read fully and end of file reached, further
>>> read() returns a -1. If a stream is re-read, it returns a -1 which
>>> is incorrect data. E.g.in the repro for DERBY-500, the update
>>> statement has multiple rows that qualify and since the stream
>>> parameter is used; the first row gets updated with the correct value
>>> and the stream is drained. For the subsequent rows, the read from the
>>> stream parameter value returns -1 and thus is updated with incorrect
>>> data.When retrieving the row back, the format of the fields is
>>> incorrect and thus the exception. __________
>>> This patch
>>>
>>> 1. adds changes to RawToBinaryFormatStream and ReaderToUTF8Stream to
>>> throw an EOFException if stream is re-read.
>>> If a stream value has been fully read and end of file reached, any
>>> further reads on the stream object will result in an EOFException.
>>> This seems reasonable and more correct than using incorrect values.
>>> Adds a new error message - 'Stream has already been read and
>>> end-of-file reached and cannot be re-used.'
>>>
>>> 2. changes to RememberBytesInputStream to keep track of the stream
>>> state and not call read on the stream objects once eof is reached.
>>>
>>> 3. Fix a bug in StoredPage.logColumn related to streams. In one
>>> particular scenario, column was not being set to
>>> RememberBytesInputStream object and thus losing the data that would
>>> be read from stream into RememberBytesInputStream.
>>>
>>> 4. adds testcases to store/streamingColumn.java and lang/forbitdata.java
>>>
>>> Also note
>>> - This fix affects cases when a stream is re-used in which case an
>>> exception will be thrown. So code that reads the stream once and
>>> materializes it will not be affected. E.g. Currently in case of
>>> char,varchar,long varchar, streams are materialized and this will
>>> work fine as before.
>>>
>>>
>>> Ran tests ok on jdk142/win2k (using classes directory)
>>>
>>> svn stat
>>> M
>>> java\engine\org\apache\derby\impl\jdbc\RawToBinaryFormatStream.java
>>> M java\engine\org\apache\derby\impl\jdbc\ReaderToUTF8Stream.java
>>> M
>>> java\engine\org\apache\derby\impl\store\raw\data\RememberBytesInputStream.java
>>>
>>> M java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java
>>> M java\engine\org\apache\derby\iapi\reference\SQLState.java
>>> M java\engine\org\apache\derby\loc\messages_en.properties
>>> M
>>> java\testing\org\apache\derbyTesting\functionTests\tests\lang\forbitdata.java
>>>
>>> M
>>> java\testing\org\apache\derbyTesting\functionTests\tests\store\streamingColumn.java
>>>
>>> M
>>> java\testing\org\apache\derbyTesting\functionTests\master\streamingColumn.out
>>>
>>> M
>>> java\testing\org\apache\derbyTesting\functionTests\master\forbitdata.out
>>>
>>> I'll add clarifications to the paper - JDBCImplementation.html and
>>> attach it as another patch to this jira entry.
>>>
>>> Can someone please review it. Thanks.
>>>
>>>
>>>
>>>
>>>
>>>> Update/Select failure when BLOB/CLOB fields updated in several rows
>>>> by PreparedStatement using setBinaryStream and setCharacterStream
>>>> -------------------------------------------------------------------------------------------------------------------------------------
>>>>
>>>>
>>>> Key: DERBY-500
>>>> URL: http://issues.apache.org/jira/browse/DERBY-500
>>>> Project: Derby
>>>> Type: Bug
>>>> Components: JDBC
>>>> Versions: 10.1.1.0
>>>> Environment: Windows 2000, java SDK 1.4
>>>> Reporter: Peter Kovgan
>>>> Assignee: Sunitha Kambhampati
>>>> Fix For: 10.1.2.0
>>>> Attachments: Derby500.diff.txt, Derby500.stat.txt
>>>>
>>>> I have table contained BLOB and CLOB fields:
>>>> Create table string is:
>>>> private static final String CREATE = "CREATE TABLE ta (" +
>>>> "ta_id INTEGER NOT NULL," +
>>>> "mname VARCHAR( 254 ) NOT NULL," +
>>>> "mvalue INT NOT NULL," +
>>>> "mdate DATE NOT NULL," +
>>>> "bytedata BLOB NOT NULL," +
>>>> "chardata CLOB NOT NULL," +
>>>> "PRIMARY KEY ( ta_id ))";
>>>> Then I insert 2000 rows in the table.
>>>> Then I update all 2000 rows by command:
>>>> private static final String UPDATE = "UPDATE ta " +
>>>> "SET bytedata=? ,chardata=? " +
>>>> "WHERE mvalue=?";
>>>> /**create blob and clob arrays**/
>>>> int len1 = 10000;//for blob length data
>>>> int len2 = 15000;//for clob length data
>>>> byte buf [] = new byte[len1];
>>>> for(int i=0;i<len1;i++){
>>>> buf [i] = (byte)45;
>>>> }
>>>> ByteArrayInputStream bais = new ByteArrayInputStream(buf);
>>>> char[] bufc = new char[len2];
>>>> for (int i = 0; i < bufc.length; i++) {
>>>> bufc[i] = (char)'b';
>>>> }
>>>> CharArrayReader car = new CharArrayReader(bufc);
>>>> /***/
>>>> PreparedStatement pstmt = connection.prepareStatement(UPDATE);
>>>> pstmt.setBinaryStream(1,bais, len1);
>>>> pstmt.setCharacterStream(2,car, len2);
>>>> pstmt.setInt(3,5000);
>>>> int updated = pstmt.executeUpdate();
>>>> pstmt.close();
>>>> System.out.printlen("updated ="+updated );
>>>> all 2000 rows updated , because I receive output : updated =2000
>>>> But If I run select (SELECT bytedata ,chardata FROM ta) after
>>>> update, select failed with error:
>>>> ERROR XSDA7: Restore of a serializable or SQLData object of class ,
>>>> attempted to
>>>> read more data than was originally stored
>>>> at
>>>> org.apache.derby.iapi.error.StandardException.newException(StandardEx
>>>> ception.java)
>>>> at
>>>> org.apache.derby.impl.store.raw.data.StoredPage.readRecordFromArray(S
>>>> toredPage.java)
>>>> at
>>>> org.apache.derby.impl.store.raw.data.StoredPage.restoreRecordFromSlot
>>>> (StoredPage.java)
>>>> at
>>>> org.apache.derby.impl.store.raw.data.BasePage.fetchFromSlot(BasePage.
>>>> java)
>>>> at
>>>> org.apache.derby.impl.store.access.conglomerate.GenericScanController
>>>> .fetchRows(GenericScanController.java)
>>>> at
>>>> org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(HeapS
>>>> can.java)
>>>> at
>>>> org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(
>>>> BulkTableScanResultSet.java)
>>>> at
>>>> org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCo
>>>> re(BulkTableScanResultSet.java)
>>>> at
>>>> org.apache.derby.impl.sql.execute.NestedLoopJoinResultSet.getNextRowC
>>>> ore(NestedLoopJoinResultSet.java)
>>>> at
>>>> org.apache.derby.impl.sql.execute.NestedLoopLeftOuterJoinResultSet.ge
>>>> tNextRowCore(NestedLoopLeftOuterJoinResultSet.java)
>>>> at
>>>> org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRow
>>>> Core(ProjectRestrictResultSet.java)
>>>> at
>>>> org.apache.derby.impl.sql.execute.SortResultSet.getRowFromResultSet(S
>>>> ortResultSet.java)
>>>> at
>>>> org.apache.derby.impl.sql.execute.SortResultSet.getNextRowFromRS(Sort
>>>> ResultSet.java)
>>>> at
>>>> org.apache.derby.impl.sql.execute.SortResultSet.loadSorter(SortResult
>>>> Set.java)
>>>> at
>>>> org.apache.derby.impl.sql.execute.SortResultSet.openCore(SortResultSe
>>>> t.java)
>>>> at
>>>> org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.open(BasicN
>>>> oPutResultSetImpl.java)
>>>> at
>>>> org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPre
>>>> paredStatement.java)
>>>> at
>>>> org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedState
>>>> ment.java)
>>>> at
>>>> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Em
>>>> bedPreparedStatement.java)
>>>> at
>>>> org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(EmbedPrepar
>>>> edStatement.java)
>>>> at com.beep_beep.dbtest.complex.Benchmark.testSelect(Unknown
>>>> Source)
>>>> at
>>>> com.beep_beep.dbtest.complex.Benchmark.executeSimplestBigTable(Unknown
>>>> Sour
>>>> ce)
>>>> at com.beep_beep.dbtest.complex.Benchmark.testBigTable(Unknown
>>>> Source)
>>>> at
>>>> com.beep_beep.dbtest.complex.Benchmark.executeDegradationBenchmark(Unknown
>>>>
>>>> Source)
>>>> at com.beep_beep.dbtest.complex.Benchmark.main(Unknown Source)
>>>>
>>>> From the stack trace and from console I see that Update passed, but
>>>> error was raised in Select after Update.
>>>
>>>
>>>
>>>> When I try the same update, but with difference(I changed WHERE
>>>> clause, causing update only 1 row):
>>>> private static final String UPDATE = "UPDATE ta " +
>>>> "SET bytedata=? ,chardata=? " +
>>>> "WHERE mname=?";
>>>> PreparedStatement pstmt = connection.prepareStatement(UPDATE);
>>>> pstmt.setBinaryStream(1,bais, len1);
>>>> pstmt.setCharacterStream(2,car, len2);
>>>> pstmt.setInt(3,"PETER");
>>>> int updated = pstmt.executeUpdate();
>>>> pstmt.close();
>>>> System.out.printlen("updated ="+updated );
>>>> Only 1 row updated , because I receive output : updated =1
>>>> In this case I have NO errors in select(the same as previous) .
>>>> My assumption:
>>>> It seems that Update receives ByteArrayInputStream and updates
>>>> correctly only 1 row, then all rows updated by some
>>>> incorrect value(may be because ByteArrayInputStream reached its end
>>>> in first update), causing select failure.
>>>> I tested PointBase by the same test and PointBase passed this stage
>>>> without errors, no matter how many rows was updated.
>>>> So I think it is a bug.
>>>> Thank you.
>>>>
>>>
>>>
>>
>>
>>
>>
>
>
--- End Message ---
