I don't know if we need to relax the security restrictions on the tool or not or just document the issue and the work around. It was disconcerting when trying to get the tool to run as I thought I had another corrupt issue. The work around was pretty easy to do once I know what the issue was.
As for using the "SYSCS_UTIL.SYSCS_EXPORT_QUERY" procedure, I had not even thought of that. I was under a time crunch to figure out if it would be possible to recover from the corruption and I was pretty much blindly following along with the comments and instructions in the https://issues.apache.org/jira/browse/DERBY-6136 tool. I even used the prebuilt JAR that was attached to the issue as I really did not know how to go about building this tool. I still don't know how to go about building the tool so maybe some documentation on how to do that would be useful. Here is what my existing IJ script looked like, so I was using just a "select" to get the data. I did not even realize I could call that procedure on the "RAW_" schema. bash-3.2# less extractdata.sql connect 'jdbc:derby:memory:db;create=true'; create type serializable external name 'java.io.Serializable' language java; call syscs_util.syscs_register_tool ( 'customTool', true, 'RawDBReader', 'CONTROL', 'RAW_', '/opt/canoga/canogaview/glassfish/databases/csemdb', null, 'CSEM', 'CSEM' ); select ID, DTYPE, OPLOCK, RELATIVE_POSITION, ALIAS_NAME, DESCRIPTION, DEVICE_NAME, PARENTENTITY_ID FROM RAW_CORE_V1.DEVICE_ENTITY; extractdata.sql (END) -----Original Message----- From: Rick Hillegas [mailto:[email protected]] Sent: Monday, August 08, 2016 10:21 AM To: [email protected] Subject: Re: Need some help in trying to figure out a corrupt database Hi Brett, I'm glad to hear that this tool was useful on an old database. Some comments inline... On 8/7/16, 5:33 PM, Bergquist, Brett wrote: > Mike, Rick, Bryan, others. There was definite corruption in one page for > one table: > > 421677 |HARDWARE > |1 |3 |NULL > > |NULL > > |Console Port > > |421672 > NULL |NULL > |0 |-1 |^@^@^@^@~$ > > |^@^@^@^@ ^@^@ > > |^@^@^@^@^@^@ > > |NULL WARNING (no SQLState): Error reading field data. Offset = 65, > length = 2, datatype = VARCHAR(64): Field 0 in record 0 on page 32325 > in file cb340.dat: java.io.EOFException: null WARNING (no SQLState): > Error reading field data. Offset = 123, length = 4, datatype = > VARCHAR(64): Field 0 in record 1 on page 32325 in file cb340.dat: > java.io.EOFException: null WARNING (no SQLState): Error reading field > data. Offset = 139, length = 4, datatype = VARCHAR(64): Field 0 in > record 2 on page 32325 in file cb340.dat: java.io.EOFException: null > WARNING (no SQLState): Error reading field data. Offset = 155, length > = 4, datatype = VARCHAR(64): Field 0 in record 3 on page 32325 in file > cb340.dat: java.io.EOFException: null WARNING (no SQLState): Error > reading field data. Offset = 171, length = 4, datatype = VARCHAR(64): > Field 0 in record 4 on page 32325 in file cb340.dat: > java.io.EOFException: null WARNING (no SQLState): Error reading field > data. Offset = 187, length = 4, datatype = VARCHAR(64): Field 0 in > record 5 on page 32325 in file cb340.dat: java.io.EOFException: null > WARNING (no SQLState): Error reading field data. Offset = 203, length > = 4, datatype = VARCHAR(64): Field 0 in record 6 on page 32325 in file > cb340.dat: java.io.EOFException: null WARNING (no SQLState): Error > reading field data. Offset = 219, length = 4, datatype = VARCHAR(64): > Field 0 in record 7 on page 32325 in file cb340.dat: > java.io.EOFException: null WARNING (no SQLState): Error reading field > data. Offset = 235, length = 4, datatype = VARCHAR(64): Field 0 in > record 8 on page 32325 in file cb340.dat: java.io.EOFException: null > WARNING (no SQLState): Error reading field data. Offset = 251, length > = 4, datatype = VARCHAR(64): Field 0 in record 9 on page 32325 in file > cb340.dat: java.io.EOFException: null WARNING (no SQLState): Error > reading field data. Offset = 267, length = 4, datatype = VARCHAR(64): > Field 0 in record 10 on page 32325 in file cb340.dat: > java.io.EOFException: null WARNING (no SQLState): Error reading field > data. Offset = 283, length = 4, datatype = VARCHAR(64): Field 0 in > record 11 on page 32325 in file cb340.dat: java.io.EOFException: null > WARNING (no SQLState): Error reading field data. Offset = 299, length > = 4, datatype = VARCHAR(64): Field 0 in record 12 on page 32325 in > file cb340.dat: java.io.EOFException: null WARNING (no SQLState): > Error reading field data. Offset = 315, length = 4, datatype = > VARCHAR(64): Field 0 in record 13 on page 32325 in file cb340.dat: > java.io.EOFException: null WARNING (no SQLState): Error reading field > data. Offset = 331, length = 4, datatype = VARCHAR(64): Field 0 in > record 14 on page 32325 in file cb340.dat: java.io.EOFException: null > WARNING (no SQLState): Error reading field data. Offset = 347, length > = 4, datatype = VARCHAR(64): Field 0 in record 15 on page 32325 in > file cb340.dat: java.io.EOFException: null WARNING (no SQLState): > Error reading field data. Offset = 363, length = 4, datatype = > VARCHAR(64): Field 0 in record 16 on page 32325 in file cb340.dat: > java.io.EOFException: null WARNING (no SQLState): Error reading field > data. Offset = 379, length = 4, datatype = VARCHAR(64): Field 0 in > record 17 on page 32325 in file cb340.dat: java.io.EOFException: null > WARNING (no SQLState): Error reading field data. Offset = 395, length > = 4, datatype = VARCHAR(64): Field 0 in record 18 on page 32325 in > file cb340.dat: java.io.EOFException: null WARNING (no SQLState): > Error reading field data. Offset = 411, length = 4, datatype = > VARCHAR(64): Field 0 in record 19 on page 32325 in file cb340.dat: > java.io.EOFException: null WARNING (no SQLState): Error reading field > data. Offset = 427, length = 4, datatype = VARCHAR(64): Field 0 in > record 20 on page 32325 in file cb340.dat: java.io.EOFException: null > WARNING (no SQLState): Error reading field data. Offset = 443, length > = 4, datatype = VARCHAR(64): Field 0 in record 21 on page 32325 in > file cb340.dat: java.io.EOFException: null WARNING (no SQLState): > Error reading field data. Offset = 459, length = 4, datatype = > VARCHAR(64): Field 0 in record 22 on page 32325 in file cb340.dat: > java.io.EOFException: null WARNING (no SQLState): Error reading field > data. Offset = 475, length = 4, datatype = VARCHAR(64): Field 0 in > record 23 on page 32325 in file cb340.dat: java.io.EOFException: null > ... > > This was extracted using the tool from: > > https://issues.apache.org/jira/browse/DERBY-6136 > > Mike, I believe your idea about retrieving all from the index when selecting > individually is probably true. > > I believe the corrupt occurred when bug > > https://issues.apache.org/jira/browse/DERBY-6879 > > was hit and one or more transaction log files were removed. Although > it is pointed out that doing so can and probably will cause > corruption, when you have a 500+ GB database with 18K transaction log > files to process at startup which will take multiple days to complete > and the most recent backup also backed up this condition, it was > unavoidable ;) > > Using the tool from > > https://issues.apache.org/jira/browse/DERBY-6136 > > I was able to dump the data, destroy the table, re-format and re-import the > data with SYSCS_UTIL.SYSCS_IMPORT_TABLE. From that I was able to determine > what appeared to been in progress and what the corruption was related to > (deletion of various rows) and was able to manually cleanup references to > those rows. > > So the tool was a life saver in this case. I did run into an issue with it > in that it tries to access conglomerate "c470.dat" this did not exist and > this mapped to SYSUSERS in other databases. This is old database that has > been through many upgrades of Derby on it. I was able to copy that > conglomerate from another newly created database just to let the tool > continue which it did. Some effort was put into making the tool respect the existing security mechanisms of the database. It should be easy to relax this particular check. We could read the database version from the property conglomerate and skip this hurdle if the version is before 10.9, the release which introduced NATIVE authentication and the SYSUSERS table. > A possible enhancement of the tool would be to output the data in a format > that can be imported by SYSCS_UTIL.SYSCS_IMPORT_TABLE. I was able to use > Perl to reformat the data for import but not having to do so would be > beneficial. What is needed beyond chaining the tool together with the SYSCS_UTIL.SYSCS_EXPORT_QUERY procedure (see http://db.apache.org/derby/docs/10.12/ref/rrefexportselectionproc.html)? Thanks, -Rick > I truly appreciate all of the help and suggestions! > >> On Aug 5, 2016, at 10:34 AM, mike matrigali<[email protected]> wrote: >> >> I agree with bryan's first suggestions, more info from derby.log >> would help. If it is possible just posting it currently might help >> (one from debug with line numbers even better). i would suggest filing a >> JIRA and posting info and moving discussion there. >> >> Some other questions/suggestions. >> o can you tell us more about the ddl, i have not used describe much. What i >> am looking mostly for is if there are any unique keys. >> o what exactly are the indexes that you have on this table? what i am >> looking for is what columns are in the indexes. >> o look at the query plans for the queries that you are running, both the >> ones that are working and the >> ones that are not. what i am looking for is if the optmizer has been >> able to run some queries as "covering" which means it gets >> all the values from the index and never goes to the base table. >> o if indexes are being used you might try using the optimizer hints to force >> index or force not index to better understand where >> corruption is. >> >> on the ordering question, it depends on the answers to the above. If >> indexes are used the ordering is going to be different >> for each. A base table scan should theoretically give same order for each >> scan as long as there is no updates happening. >> >> does anyone know if describe prints the columns in the order they are >> actually stored in the db? >> >> if you have unique keys and the indexes are good, there may be some ways to >> better get at the data. >> >> /mikem >> >> On 7/27/2016 7:16 AM, Bergquist, Brett wrote: >>> I wrote a tool to call SYSCS_UTIL.SYSCS_CHECK_TABLE on each table in the >>> database and not just stop on error. >>> >>> I have a database returns the error: >>> >>> Checking CORE_V1.DEVICE_ENTITY failed with exception: >>> Restore of a serializable or SQLData object of class , attempted to >>> read more data than was originally stored >>> >>> The table has 1261157 rows in it. I can query each of the columns in the >>> table individually like: >>> >>> SELECT ID FROM CORE_V1.DEVICE_ENTITY; >>> SELECT DTYPE FROM CORE_V1.DEVICE_ENTITY; >>> ... >>> SELECT DESCRIPTION FROM CORE_V1.DEVICE_ENTITY; >>> >>> These all return 1261157 rows for each select result. >>> >>> If I select multiple columns such as >>> >>> SELECT ID, DTYPE FROM CORE_V1.DEVICE_ENTITY; >>> >>> Then I get: >>> >>> ERROR XSDA7: Restore of a serializable or SQLData object of class , >>> attempted to read more data than was originally stored ERROR XJ001: Java >>> exception: ': java.io.EOFException'. >>> ij> >>> >>> The table schema looks like: >>> >>> ij> describe core_v1.device_entity; >>> COLUMN_NAME >>> |TYPE_NAME|DEC&|NUM&|COLUM&|COLUMN_DEF|CHAR_OCTE&|IS_NULL& >>> ------------------------------------------------------------------------------ >>> DTYPE |VARCHAR |NULL|NULL|64 |NULL |128 |YES >>> OPLOCK |INTEGER |0 |10 |10 |0 |NULL |NO >>> RELATIVE_POSITION |SMALLINT |0 |10 |5 |NULL |NULL |YES >>> ALIAS_NAME |VARCHAR |NULL|NULL|255 |NULL |510 |YES >>> DESCRIPTION |VARCHAR |NULL|NULL|255 |NULL |510 |YES >>> DEVICE_NAME |VARCHAR |NULL|NULL|255 |NULL |510 |YES >>> PARENTENTITY_ID |INTEGER |0 |10 |10 |NULL |NULL |YES >>> ID |INTEGER |0 |10 |10 |NULL |NULL |NO >>> >>> 8 rows selected >>> >>> I am trying to figure out how to recover and correct this table. Once >>> thought is to query each column separately and then join the output >>> externally back together and re-import the data. I wonder however if I >>> query each column if the order of the rows returned will be the same. This >>> will be necessary because I cannot select the ID with the other column to >>> identify the rows. >>> >>> Any thoughts will be most welcome. >>> >>> >>> Canoga Perkins >>> 20600 Prairie Street >>> Chatsworth, CA 91311 >>> (818) 718-6300 >>> >>> This e-mail and any attached document(s) is confidential and is intended >>> only for the review of the party to whom it is addressed. If you have >>> received this transmission in error, please notify the sender immediately >>> and discard the original message and any attachment(s). >>> . >>> >> >> -- >> email: Mike Matrigali - [email protected] >> linkedin: https://www.linkedin.com/in/MikeMatrigali >> > > Canoga Perkins > 20600 Prairie Street > Chatsworth, CA 91311 > (818) 718-6300 > > This e-mail and any attached document(s) is confidential and is intended only > for the review of the party to whom it is addressed. If you have received > this transmission in error, please notify the sender immediately and discard > the original message and any attachment(s). > Canoga Perkins 20600 Prairie Street Chatsworth, CA 91311 (818) 718-6300 This e-mail and any attached document(s) is confidential and is intended only for the review of the party to whom it is addressed. If you have received this transmission in error, please notify the sender immediately and discard the original message and any attachment(s).
