I have reviewed, and run successful tests against this patch. It has been committed to trunk as: 326307.
Sunitha Kambhampati (JIRA) wrote: > [ http://issues.apache.org/jira/browse/DERBY-500?page=all ] > > Sunitha Kambhampati updated DERBY-500: > -------------------------------------- > > Attachment: Derby500.diff.txt > > Background : > In Derby, when a stream is set as a parameter value, the wrapper stream > object used for character data is ReaderToUTF8Stream > and for binary data it is RawToBinaryFormatStream.Both these stream objects > on read() return data in a format that is used to store the respective > datatype value. E.g in case of char, the characters read from the user stream > are converted using utf-8 derby specific encoding and read calls return > the data as expected by store layer. Beginning 2 bytes either have the utflen > or has zeroes, or if it is a long string, then the value is ended with the > special marker 0xE0 , 0x00, 0x00. For binary data, the stream data is > prepended with 4 zeroes. > > Problem: > once,the stream has been read fully and end of file reached, further read() > returns a -1. If a stream is re-read, it returns a -1 which is incorrect > data. E.g.in the repro for DERBY-500, the update statement has multiple rows > that qualify and since the stream parameter is used; the first row gets > updated with the correct value and the stream is drained. For the subsequent > rows, the read from the stream parameter value returns -1 and thus is updated > with incorrect data.When retrieving the row back, the format of the fields is > incorrect and thus the exception. > __________ > This patch > > 1. adds changes to RawToBinaryFormatStream and ReaderToUTF8Stream to throw an > EOFException if stream is re-read. > If a stream value has been fully read and end of file reached, any further > reads on the stream object will result in an EOFException. This seems > reasonable and more correct than using incorrect values. > Adds a new error message - 'Stream has already been read and end-of-file > reached and cannot be re-used.' > > 2. changes to RememberBytesInputStream to keep track of the stream state and > not call read on the stream objects once eof is reached. > > 3. Fix a bug in StoredPage.logColumn related to streams. In one particular > scenario, column was not being set to RememberBytesInputStream object and > thus losing the data that would be read from stream into > RememberBytesInputStream. > > 4. adds testcases to store/streamingColumn.java and lang/forbitdata.java > > > Also note > - This fix affects cases when a stream is re-used in which case an exception > will be thrown. > So code that reads the stream once and materializes it will not be affected. > E.g. Currently in case of char,varchar,long varchar, streams are > materialized and this will work fine as before. > > > Ran tests ok on jdk142/win2k (using classes directory) > > svn stat > M java\engine\org\apache\derby\impl\jdbc\RawToBinaryFormatStream.java > M java\engine\org\apache\derby\impl\jdbc\ReaderToUTF8Stream.java > M > java\engine\org\apache\derby\impl\store\raw\data\RememberBytesInputStream.java > M java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java > M java\engine\org\apache\derby\iapi\reference\SQLState.java > M java\engine\org\apache\derby\loc\messages_en.properties > M > java\testing\org\apache\derbyTesting\functionTests\tests\lang\forbitdata.java > M > java\testing\org\apache\derbyTesting\functionTests\tests\store\streamingColumn.java > M > java\testing\org\apache\derbyTesting\functionTests\master\streamingColumn.out > M > java\testing\org\apache\derbyTesting\functionTests\master\forbitdata.out > > I'll add clarifications to the paper - JDBCImplementation.html and attach it > as another patch to this jira entry. > > Can someone please review it. Thanks. > > > >>Update/Select failure when BLOB/CLOB fields updated in several rows by >>PreparedStatement using setBinaryStream and setCharacterStream >>------------------------------------------------------------------------------------------------------------------------------------- >> >> Key: DERBY-500 >> URL: http://issues.apache.org/jira/browse/DERBY-500 >> Project: Derby >> Type: Bug >> Components: JDBC >> Versions: 10.1.1.0 >> Environment: Windows 2000, java SDK 1.4 >> Reporter: Peter Kovgan >> Assignee: Sunitha Kambhampati >> Fix For: 10.1.2.0 >> Attachments: Derby500.diff.txt, Derby500.stat.txt >> >>I have table contained BLOB and CLOB fields: >>Create table string is: >>private static final String CREATE = "CREATE TABLE ta (" + >> "ta_id INTEGER NOT NULL," + >> "mname VARCHAR( 254 ) NOT NULL," + >> "mvalue INT NOT NULL," + >> "mdate DATE NOT NULL," + >> "bytedata BLOB NOT NULL," + >> "chardata CLOB NOT NULL," + >> "PRIMARY KEY ( ta_id ))"; >>Then I insert 2000 rows in the table. >>Then I update all 2000 rows by command: >>private static final String UPDATE = "UPDATE ta " + >> "SET bytedata=? ,chardata=? " + >> "WHERE mvalue=?"; >>/**create blob and clob arrays**/ >> int len1 = 10000;//for blob length data >> int len2 = 15000;//for clob length data >> byte buf [] = new byte[len1]; >> for(int i=0;i<len1;i++){ >> buf [i] = (byte)45; >> } >> ByteArrayInputStream bais = new ByteArrayInputStream(buf); >> >> char[] bufc = new char[len2]; >> for (int i = 0; i < bufc.length; i++) { >> bufc[i] = (char)'b'; >> } >> CharArrayReader car = new CharArrayReader(bufc); >>/***/ >>PreparedStatement pstmt = connection.prepareStatement(UPDATE); >>pstmt.setBinaryStream(1,bais, len1); >>pstmt.setCharacterStream(2,car, len2); >>pstmt.setInt(3,5000); >>int updated = pstmt.executeUpdate(); >>pstmt.close(); >>System.out.printlen("updated ="+updated ); >>all 2000 rows updated , because I receive output : updated =2000 >>But If I run select (SELECT bytedata ,chardata FROM ta) after update, >>select failed with error: >>ERROR XSDA7: Restore of a serializable or SQLData object of class , attempted >>to >> read more data than was originally stored >> at >> org.apache.derby.iapi.error.StandardException.newException(StandardEx >>ception.java) >> at >> org.apache.derby.impl.store.raw.data.StoredPage.readRecordFromArray(S >>toredPage.java) >> at >> org.apache.derby.impl.store.raw.data.StoredPage.restoreRecordFromSlot >>(StoredPage.java) >> at >> org.apache.derby.impl.store.raw.data.BasePage.fetchFromSlot(BasePage. >>java) >> at >> org.apache.derby.impl.store.access.conglomerate.GenericScanController >>.fetchRows(GenericScanController.java) >> at >> org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(HeapS >>can.java) >> at >> org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray( >>BulkTableScanResultSet.java) >> at >> org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCo >>re(BulkTableScanResultSet.java) >> at >> org.apache.derby.impl.sql.execute.NestedLoopJoinResultSet.getNextRowC >>ore(NestedLoopJoinResultSet.java) >> at >> org.apache.derby.impl.sql.execute.NestedLoopLeftOuterJoinResultSet.ge >>tNextRowCore(NestedLoopLeftOuterJoinResultSet.java) >> at >> org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRow >>Core(ProjectRestrictResultSet.java) >> at >> org.apache.derby.impl.sql.execute.SortResultSet.getRowFromResultSet(S >>ortResultSet.java) >> at >> org.apache.derby.impl.sql.execute.SortResultSet.getNextRowFromRS(Sort >>ResultSet.java) >> at >> org.apache.derby.impl.sql.execute.SortResultSet.loadSorter(SortResult >>Set.java) >> at >> org.apache.derby.impl.sql.execute.SortResultSet.openCore(SortResultSe >>t.java) >> at >> org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.open(BasicN >>oPutResultSetImpl.java) >> at >> org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPre >>paredStatement.java) >> at >> org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedState >>ment.java) >> at >> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Em >>bedPreparedStatement.java) >> at >> org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(EmbedPrepar >>edStatement.java) >> at com.beep_beep.dbtest.complex.Benchmark.testSelect(Unknown Source) >> at >> com.beep_beep.dbtest.complex.Benchmark.executeSimplestBigTable(Unknown Sour >>ce) >> at com.beep_beep.dbtest.complex.Benchmark.testBigTable(Unknown Source) >> at >> com.beep_beep.dbtest.complex.Benchmark.executeDegradationBenchmark(Unknown >>Source) >> at com.beep_beep.dbtest.complex.Benchmark.main(Unknown Source) >>From the stack trace and from console I see that Update passed, but error was >>raised in Select after Update. >>When I try the same update, but with difference(I changed WHERE clause, >>causing update only 1 row): >>private static final String UPDATE = "UPDATE ta " + >> "SET bytedata=? ,chardata=? " + >> "WHERE mname=?"; >>PreparedStatement pstmt = connection.prepareStatement(UPDATE); >>pstmt.setBinaryStream(1,bais, len1); >>pstmt.setCharacterStream(2,car, len2); >>pstmt.setInt(3,"PETER"); >>int updated = pstmt.executeUpdate(); >>pstmt.close(); >>System.out.printlen("updated ="+updated ); >>Only 1 row updated , because I receive output : updated =1 >>In this case I have NO errors in select(the same as previous) . >>My assumption: >>It seems that Update receives ByteArrayInputStream and updates correctly only >>1 row, then all rows updated by some >>incorrect value(may be because ByteArrayInputStream reached its end in first >>update), causing select failure. >>I tested PointBase by the same test and PointBase passed this stage without >>errors, no matter how many rows was updated. >>So I think it is a bug. >>Thank you. > >
