[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13799743#comment-13799743 ] ASF subversion and git services commented on DERBY-5234: Commit 1533682 from [~kmarsden] in branch 'code/branches/10.5' [ https://svn.apache.org/r1533682 ] DERBY-6382 fter Inplace compress: java.sql.SQLException: Page Page(10219,Container(0, 1136)) could not be read from disk caused by Caused by: java.io.EOFException: Reached end of file while attempting to read a whole page. This is a partial fix for DERBY-5234. Merged revision 1337258 from 10.8 Contributed by Rick Hillegas > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_10, derby_triage10_9 > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, > derby-5234-01-aa-emptyAllocPage.diff, derby-5234-01-ab-emptyAllocPage.diff, > derby-5234-02-aa-edgeCaseTests.diff, Derby5234.java, Derby5234.java, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addP
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13799256#comment-13799256 ] ASF subversion and git services commented on DERBY-5234: Commit 1533540 from [~kmarsden] in branch 'code/branches/10.6' [ https://svn.apache.org/r1533540 ] DERBY-6382 fter Inplace compress: java.sql.SQLException: Page Page(10219,Container(0, 1136)) could not be read from disk caused by Caused by: java.io.EOFException: Reached end of file while attempting to read a whole page. This is a partial fix for DERBY-5234. Merged revision 1337258 from 10.8 Contributed by Rick Hillegas > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_10, derby_triage10_9 > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, > derby-5234-01-aa-emptyAllocPage.diff, derby-5234-01-ab-emptyAllocPage.diff, > derby-5234-02-aa-edgeCaseTests.diff, Derby5234.java, Derby5234.java, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addP
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13799058#comment-13799058 ] ASF subversion and git services commented on DERBY-5234: Commit 1533412 from [~kmarsden] in branch 'code/branches/10.7' [ https://svn.apache.org/r1533412 ] DERBY-6382 fter Inplace compress: java.sql.SQLException: Page Page(10219,Container(0, 1136)) could not be read from disk caused by Caused by: java.io.EOFException: Reached end of file while attempting to read a whole page. This is a partial fix for DERBY-5234. Merged revision 1337258 from 10.8 Contributed by Rick Hillegas > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_10, derby_triage10_9 > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, > derby-5234-01-aa-emptyAllocPage.diff, derby-5234-01-ab-emptyAllocPage.diff, > derby-5234-02-aa-edgeCaseTests.diff, Derby5234.java, Derby5234.java, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addP
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13798572#comment-13798572 ] Kathey Marsden commented on DERBY-5234: --- So, although it was closed CannotReproduce, I think *something* was fixed in this issue. 1335570 and 1335677 were committed to trunk and ported to 10.8 branch at subversion revision 1337258. For bookkeeping and possible further backport it would seem to make sense to open a separate issue for what was fixed. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_10, derby_triage10_9 > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, > derby-5234-01-aa-emptyAllocPage.diff, derby-5234-01-ab-emptyAllocPage.diff, > derby-5234-02-aa-edgeCaseTests.diff, Derby5234.java, Derby5234.java, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13556148#comment-13556148 ] Rick Hillegas commented on DERBY-5234: -- Knut's understanding of the bug agrees with mine. There seems to be consensus that the table compression code gives rise to many data corruptions and that code needs to be overhauled. That's why this issue was linked to DERBY-5876, an issue which describes a systemic rather than piecemeal mini-project for re-writing that part of Derby. Thanks. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_10, derby_triage10_9 > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, > derby-5234-01-aa-emptyAllocPage.diff, derby-5234-01-ab-emptyAllocPage.diff, > derby-5234-02-aa-edgeCaseTests.diff, Derby5234.java, Derby5234.java, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.a
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13556022#comment-13556022 ] Knut Anders Hatlen commented on DERBY-5234: --- The state of this bug report is a bit confusing, since it contains multiple reports of issues with similar symptoms. Those issues may or may not be related. One fix has been checked in and seems to have fixed one of the issues, but it also sounds like at least one of the problems remains unsolved. My reading of the comments on this bug report is that the issue Stefan saw could be reproduced, and it has been fixed. That fix is included in the 10.8.3.0 maintenance release. However, the problems reported by Varma and Saurabh have not been reproduced by anyone else yet. Saurabh has tested the fix and concluded that the fix for Stefan's issue did not fix the issue he saw. It is not known whether or not the fix addresses the problem reported by Varma. Does that accurately describe the current state of the bug report? > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_10, derby_triage10_9 > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, > derby-5234-01-aa-emptyAllocPage.diff, derby-5234-01-ab-emptyAllocPage.diff, > derby-5234-02-aa-edgeCaseTests.diff, Derby5234.java, Derby5234.java, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13555975#comment-13555975 ] Stefan Sitte commented on DERBY-5234: - I am very disappointed that even after one year you know this error and have a repro you closed the ticket. I'm wondering why the problems regarding the compression (inplace and normal) won't be fixed. In my opinion this is essential for running a database. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_10, derby_triage10_9 > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, > derby-5234-01-aa-emptyAllocPage.diff, derby-5234-01-ab-emptyAllocPage.diff, > derby-5234-02-aa-edgeCaseTests.diff, Derby5234.java, Derby5234.java, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapContro
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286706#comment-13286706 ] Saurabh Singhi commented on DERBY-5234: --- I have given up as well. Resorting to dynamic creation and deletion of tables (as opposed to rows) for now. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.1.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[
https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281224#comment-13281224
]
Saurabh Singhi commented on DERBY-5234:
---
Hi Rick,
My program created a new 10.8.2.3 database along with the regular select,
delete, lock, compress and insert statements.
As per your suggestions, the program executed a LOCK statement (EXCLUSIVE lock)
before executing the SYSCS_COMPRESS_TABLE call.
I'm still getting the deadlock exception (while selecting and inserting) shown
below:-
***
22 May 2012 13:37:32 ProcessDAO.getLastRecordFromIntervalTableShowGap = A lock
could not be obtained due to a deadlock, cycle of locks and waiters is:
Lock : TABLE, PROCESS_055234_HR, Tablelock
Waiting XID : {104215, IS} , APP, SELECT CpuNum, SampleTimestamp AS
TimeStamp, ProcBusyPercent AS BusyPercent FROM PROCESS_055234_HR WHERE
SampleTimestamp >= ? AND Volume=? AND Subvol=? AND Filename=? AND
ProcessName=? AND CpuNum=? AND PIN=? ORDER BY SampleTimestamp
Granted XID : {102377, X}
Lock : ROW, SYSCONGLOMERATES, (7,9)
Waiting XID : {102377, X} , APP, alter table "APP"."PROCESS_055234_HR"
compress sequential
Granted XID : {102377, S} , {104215, S}
. The selected victim is XID : 104215.
java.sql.SQLTransactionRollbackException: A lock could not be obtained due to a
deadlock, cycle of locks and waiters is:
Lock : TABLE, PROCESS_055234_HR, Tablelock
Waiting XID : {104215, IS} , APP, SELECT CpuNum, SampleTimestamp AS
TimeStamp, ProcBusyPercent AS BusyPercent FROM PROCESS_055234_HR WHERE
SampleTimestamp >= ? AND Volume=? AND Subvol=? AND Filename=? AND
ProcessName=? AND CpuNum=? AND PIN=? ORDER BY SampleTimestamp
Granted XID : {102377, X}
Lock : ROW, SYSCONGLOMERATES, (7,9)
Waiting XID : {102377, X} , APP, alter table "APP"."PROCESS_055234_HR"
compress sequential
Granted XID : {102377, S} , {104215, S}
. The selected victim is XID : 104215.
at
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:89)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Util.java:256)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:424)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(TransactionResourceImpl.java:353)
at
org.apache.derby.impl.jdbc.EmbedConnection.handleException(EmbedConnection.java:2290)
at
org.apache.derby.impl.jdbc.ConnectionChild.handleException(ConnectionChild.java:82)
at
org.apache.derby.impl.jdbc.EmbedPreparedStatement.(EmbedPreparedStatement.java:151)
at
org.apache.derby.impl.jdbc.EmbedPreparedStatement20.(EmbedPreparedStatement20.java:82)
at
org.apache.derby.impl.jdbc.EmbedPreparedStatement30.(EmbedPreparedStatement30.java:63)
at
org.apache.derby.impl.jdbc.EmbedPreparedStatement40.(EmbedPreparedStatement40.java:40)
at
org.apache.derby.jdbc.Driver40.newEmbedPreparedStatement(Driver40.java:107)
at
org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(EmbedConnection.java:1615)
at
org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(EmbedConnection.java:1443)
at
com.idelji.operations.iasset.plugin.imon.dao.ProcessDAO.getLastRecordFromIntervalTableShowGap(ProcessDAO.java:503)
at
imon.utils.DataSourceRouter.getLatestProcessData(DataSourceRouter.java:608)
at
imon.db.ProcessDataProvider.getLatestData(ProcessDataProvider.java:131)
at
imon.charts.LineChartWithSlider.refreshChartData(LineChartWithSlider.java:558)
at
imon.charts.LineChartWithSlider$1.handle(LineChartWithSlider.java:327)
at
imon.charts.LineChartWithSlider$1.handle(LineChartWithSlider.java:322)
at
com.sun.scenario.animation.shared.TimelineClipCore.visitKeyFrame(TimelineClipCore.java:196)
at
com.sun.scenario.animation.shared.TimelineClipCore.playTo(TimelineClipCore.java:137)
at javafx.animation.Timeline.impl_playTo(Timeline.java:163)
at
com.sun.scenario.animation.shared.InfiniteClipEnvelope.timePulse(InfiniteClipEnvelope.java:89)
at javafx.animation.Animation.impl_timePulse(Animation.java:952)
at
com.sun.scenario.animation.shared.AnimationPulseReceiver.timePulse(AnimationPulseReceiver.java:75)
at
com.sun.scenario.animation.AbstractMasterTimer.timePulseImpl(AbstractMasterTimer.java:357)
at
com.sun.scenario.animation.AbstractMasterTimer$MainLoop.run(AbstractMasterTimer.java:280)
at
com.sun.javafx.tk.quantum.QuantumToolkit.pulse(QuantumToolkit.java:381)
at
com.sun.javafx.tk.quantum.QuantumToolkit$8.run(QuantumToolkit.java:311)
at com.sun.glass.ui.win.WinApplication._
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281124#comment-13281124 ] Saurabh Singhi commented on DERBY-5234: --- Hi Rick, The "something" that went wrong was the same container related exception that this thread is all about. And the new experiment that I'm soon going to try out would be just as you suggested. Thanks. Saurabh > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281118#comment-13281118 ] Rick Hillegas commented on DERBY-5234: -- Hi Saurabh, Just want to make sure we're not talking past one another. The experiment (using a new, 10.8.2.3 database) would be to LOCK the table before compressing the table with SYSCS_COMPRESS_TABLE. On 2012-05-12 you said that you tried that experiment and something went wrong. It would be good to understand what went wrong. Thanks, -Rick > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281109#comment-13281109 ] Saurabh Singhi commented on DERBY-5234: --- Hi Rick, I tried locking the corrupted table but it did not yield any different results. Should I include a LOCK statement with the tables created and populated with the new 10.8 trunk and inform you of the results? > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.sto
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278039#comment-13278039 ] Rick Hillegas commented on DERBY-5234: -- Thanks, Saurabh. Can you describe what went wrong when you locked the table before compressing it? Thanks. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278036#comment-13278036 ] Saurabh Singhi commented on DERBY-5234: --- Yes Rick, I have tried acquiring an exclusive lock before issuing the SYSCS_COMPRESS_TABLE, but that did not quite work either. Thanks, Saurabh > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at >
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278025#comment-13278025 ] Rick Hillegas commented on DERBY-5234: -- Thanks, Saurabh. I see that DERBY-4275 is fixed in the trunk and in the 10.8 branch. DERBY-5351 was closed as a duplicate of DERBY-4275. However, it looks like DERBY-3683 and DERBY-5358 are still open. Have you tried issuing a LOCK TABLE command to seize exclusive access to the table before you issue SYSCS_COMPRESS_TABLE? This will throttle your concurrency every time you compress the table, but that is better than corrupting the table. Thanks, -Rick > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown >
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278009#comment-13278009 ] Saurabh Singhi commented on DERBY-5234: --- Rick, I was using SYSCS_COMPRESS_TABLE originally, but that lead to: DERBY-4275,DERBY-5351, DERBY-3683 & DERBY-5358, so I had to switch to the INPLACE version. Thanks, Saurabh > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(U
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13277936#comment-13277936 ] Rick Hillegas commented on DERBY-5234: -- Hi Saurabh, Have you tried running SYSCS_COMPRESS_TABLE rather than SYSCS_INPLACE_COMPRESS_TABLE? On 2011-05-18, Varma reported that SYSCS_COMPRESS_TABLE improved the situation and may have fixed the corrupted table. Thanks, -Rick > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13277899#comment-13277899 ] Saurabh Singhi commented on DERBY-5234: --- Hi Rick, I created and ran an ij script against my database, but that does not reproduces the problem. I'm not really sure what else to do here. Thanks, Saurabh. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Sourc
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276167#comment-13276167 ] Saurabh Singhi commented on DERBY-5234: --- Hi Rick, Actually, I did not know about an ij script until you just mentioned. I'll create one and upload here. Also, I'll try to isolate the code that is making the calls and possibly upload that too here along with my db. One problem in doing so is that the application is doing delete, compress and insert operations simultaneously and it might be hard to separate the code, but I'll see if I can do that so that the bug can be reproduced at your end. Also, I did try calling the INPLACE command with all combinations and it did not have any effect. Thanks. Saurabh > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[
https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276158#comment-13276158
]
Rick Hillegas commented on DERBY-5234:
--
Hi Saurabh,
By scripting, I mean: can you attach a program or ij script which causes this
corruption? That will allow us to reproduce the problem in the laboratory.
Have you considered setting the other trailing arguments of the compression
procedure, like so:
CALL SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE('APP', 'TABLENAME', 1, 1, 1)
This will cause the compression to take longer because Derby will do a more
thorough job of garbage-collecting the empty space. That may take Derby down
code paths which work around this bug. It would be interesting to know whether
the bug occurs when you specify this extra garbage collection. To run this
experiment you will again need to start with a new, uncorrupted database
created by 10.8.2.3.
Thanks,
-Rick
> Unable to insert data into table. Failed due be "ERROR XSDG0: Page
> Page(51919,Container(0, 1104)) could not be read from disk."
> ---
>
> Key: DERBY-5234
> URL: https://issues.apache.org/jira/browse/DERBY-5234
> Project: Derby
> Issue Type: Bug
> Components: Store
>Affects Versions: 10.5.3.0
> Environment: HP-UX 11iv2 in production environment with JDK1.6;
> Solaris 5/10 in test environment with JDK 1.6
>Reporter: Varma R
>Assignee: Rick Hillegas
>Priority: Critical
> Labels: ERROR, XSDG0, apache, corruption, data, derby,
> derby_triage10_9
> Fix For: 10.8.2.3, 10.9.0.0
>
> Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out,
> DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java,
> Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff,
> derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff,
> log191.dat, log85.dat
>
>
> One of the derby database table "gets corrupted"/"indicates connection not
> available" during processing inserts from java client application as shown in
> the trace and the only way to recover from this error is to rebuild the DB -
> by deleting the data and creating the tables again. This happens once in a
> while (thrice in a span of two months) and the java application (run in
> multiple servers), which updates the database, processes around 100 million
> transactions per hour (in total and each transation results in 4-5 updates to
> the DB)
> There are eight tables in the derby database.
>TABLE NAME ROWS COUNT (at time of corruption)
> -
>KPI.KPI_MERGEIN; 362917
>KPI.KPI_IN; 422508
>KPI.KPI_DROPPED;53667
>KPI.KPI_ERROR1; 0
>KPI.KPI_ERROR2; 2686
>KPI.KPI_ERRORMERGE;0
>KPI.KPI_MERGEOUT; 362669
>KPI.KPI_OUT; 125873
> The derby database has been started with the following parameters
> CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true
> -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5
> -Dderby.locks.waitTimeout=
> -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M
> -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start
> -h $KPIDERBYHOST -p $DERBY_KPI_PORT"
> The corrupted database tar (filesystem) in live environment was moved to a
> test system (Solaris system) and few checks were run on the corrupted DB as
> part of analysis (DB does start fine)
> While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is
> successful. But when a new row is inserted into KPI.KPI_MERGEIN table using
> command line tool it's throwing below error message (the same message that
> appeared in live
> ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR,
> A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD');
> ERROR 08006: A network protocol error was encountered and the connection has
> been terminated: the requested command encountered an unarchitected and
> implementation-specific condition for which there was no architected message
> and in derby.log file it shows below error stacktrace.
> ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown
> Source)
> at
> org.apache.derby.impl.store.raw.dat
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[
https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276150#comment-13276150
]
Saurabh Singhi commented on DERBY-5234:
---
Hi Rick,
I still cannot get past the problem.
I'm using the command : CALL SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE('APP',
'TABLENAME', 0, 0, 1) .
I don't quite understand what do you mean by "scripting" it? Please elaborate.
Thanks.
-Saurabh
> Unable to insert data into table. Failed due be "ERROR XSDG0: Page
> Page(51919,Container(0, 1104)) could not be read from disk."
> ---
>
> Key: DERBY-5234
> URL: https://issues.apache.org/jira/browse/DERBY-5234
> Project: Derby
> Issue Type: Bug
> Components: Store
>Affects Versions: 10.5.3.0
> Environment: HP-UX 11iv2 in production environment with JDK1.6;
> Solaris 5/10 in test environment with JDK 1.6
>Reporter: Varma R
>Assignee: Rick Hillegas
>Priority: Critical
> Labels: ERROR, XSDG0, apache, corruption, data, derby,
> derby_triage10_9
> Fix For: 10.8.2.3, 10.9.0.0
>
> Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out,
> DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java,
> Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff,
> derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff,
> log191.dat, log85.dat
>
>
> One of the derby database table "gets corrupted"/"indicates connection not
> available" during processing inserts from java client application as shown in
> the trace and the only way to recover from this error is to rebuild the DB -
> by deleting the data and creating the tables again. This happens once in a
> while (thrice in a span of two months) and the java application (run in
> multiple servers), which updates the database, processes around 100 million
> transactions per hour (in total and each transation results in 4-5 updates to
> the DB)
> There are eight tables in the derby database.
>TABLE NAME ROWS COUNT (at time of corruption)
> -
>KPI.KPI_MERGEIN; 362917
>KPI.KPI_IN; 422508
>KPI.KPI_DROPPED;53667
>KPI.KPI_ERROR1; 0
>KPI.KPI_ERROR2; 2686
>KPI.KPI_ERRORMERGE;0
>KPI.KPI_MERGEOUT; 362669
>KPI.KPI_OUT; 125873
> The derby database has been started with the following parameters
> CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true
> -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5
> -Dderby.locks.waitTimeout=
> -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M
> -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start
> -h $KPIDERBYHOST -p $DERBY_KPI_PORT"
> The corrupted database tar (filesystem) in live environment was moved to a
> test system (Solaris system) and few checks were run on the corrupted DB as
> part of analysis (DB does start fine)
> While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is
> successful. But when a new row is inserted into KPI.KPI_MERGEIN table using
> command line tool it's throwing below error message (the same message that
> appeared in live
> ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR,
> A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD');
> ERROR 08006: A network protocol error was encountered and the connection has
> been terminated: the requested command encountered an unarchitected and
> implementation-specific condition for which there was no architected message
> and in derby.log file it shows below error stacktrace.
> ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown
> Source)
> at
> org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source)
> at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown
> Source)
> at
> org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source)
> at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown
> Source)
> at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown
> Source)
> at
> org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown
> Source)
> a
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[
https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276076#comment-13276076
]
Rick Hillegas commented on DERBY-5234:
--
Hi Saurabh,
Are you able to script the problem you are seeing? I am unable to reproduce it.
Both of the repros attached to this issue now run correctly for me, even when I
boost the number of rows by a factor of 10.
What data compression command are you using? The repros use this command:
CALL SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE('APP', 'TABLENAME', 0, 0, 1)
Thanks,
-Rick
> Unable to insert data into table. Failed due be "ERROR XSDG0: Page
> Page(51919,Container(0, 1104)) could not be read from disk."
> ---
>
> Key: DERBY-5234
> URL: https://issues.apache.org/jira/browse/DERBY-5234
> Project: Derby
> Issue Type: Bug
> Components: Store
>Affects Versions: 10.5.3.0
> Environment: HP-UX 11iv2 in production environment with JDK1.6;
> Solaris 5/10 in test environment with JDK 1.6
>Reporter: Varma R
>Assignee: Rick Hillegas
>Priority: Critical
> Labels: ERROR, XSDG0, apache, corruption, data, derby,
> derby_triage10_9
> Fix For: 10.8.2.3, 10.9.0.0
>
> Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out,
> DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java,
> Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff,
> derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff,
> log191.dat, log85.dat
>
>
> One of the derby database table "gets corrupted"/"indicates connection not
> available" during processing inserts from java client application as shown in
> the trace and the only way to recover from this error is to rebuild the DB -
> by deleting the data and creating the tables again. This happens once in a
> while (thrice in a span of two months) and the java application (run in
> multiple servers), which updates the database, processes around 100 million
> transactions per hour (in total and each transation results in 4-5 updates to
> the DB)
> There are eight tables in the derby database.
>TABLE NAME ROWS COUNT (at time of corruption)
> -
>KPI.KPI_MERGEIN; 362917
>KPI.KPI_IN; 422508
>KPI.KPI_DROPPED;53667
>KPI.KPI_ERROR1; 0
>KPI.KPI_ERROR2; 2686
>KPI.KPI_ERRORMERGE;0
>KPI.KPI_MERGEOUT; 362669
>KPI.KPI_OUT; 125873
> The derby database has been started with the following parameters
> CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true
> -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5
> -Dderby.locks.waitTimeout=
> -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M
> -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start
> -h $KPIDERBYHOST -p $DERBY_KPI_PORT"
> The corrupted database tar (filesystem) in live environment was moved to a
> test system (Solaris system) and few checks were run on the corrupted DB as
> part of analysis (DB does start fine)
> While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is
> successful. But when a new row is inserted into KPI.KPI_MERGEIN table using
> command line tool it's throwing below error message (the same message that
> appeared in live
> ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR,
> A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD');
> ERROR 08006: A network protocol error was encountered and the connection has
> been terminated: the requested command encountered an unarchitected and
> implementation-specific condition for which there was no architected message
> and in derby.log file it shows below error stacktrace.
> ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown
> Source)
> at
> org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source)
> at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown
> Source)
> at
> org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source)
> at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown
> Source)
> at org.apache.derby.impl.store.raw.data.BaseContainer.
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
Saurabh Singhi commented on DERBY-5234 Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk." Hi Rick, I'm running 10.8.2.3 (rev. 1338623). The tables were created using this revision and the problem still persists. Thanks. Best, Saurabh This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274834#comment-13274834 ] Rick Hillegas commented on DERBY-5234: -- Hi Saurabh, Derby5234.java is just a test case. It is not checked into the Derby codeline anywhere. You have the patched version of 10.8.2.3 provided that the revision stamp on derby.jar is bigger than revision 1337258. What do you see when you run sysinfo on your 10.8.2.3 installation: java -jar lib/derbyrun.jar sysinfo Thanks, -Rick > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > o
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274817#comment-13274817 ] Saurabh Singhi commented on DERBY-5234: --- Hi Brett, Actually, I do have the "Limit rows to 100" option disabled, so the DB is returning me all the rows. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.acce
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274813#comment-13274813 ] Saurabh Singhi commented on DERBY-5234: --- Hi Rick, I will give it a a try and let you know. Thanks. Best, Saurabh > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLoca
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274805#comment-13274805 ] Brett Bergquist commented on DERBY-5234: Hi Saurabh,, Could it be the you have the SquirrelSQL client option turned off that retrieves only a limited set of rows (by default 100) and are returning many rows which is then causing the client to crash with the OutOfMemory and not the Derby engine crashing? Just a thought as I know that I have caused SquirrelSQL to do this when I have turned this option off and retrieved 10's of thousands of rows. The tool tries to create a Swing table to display the rows and this causes the OutOfMemory condition in the tool, not the database engine. Brett > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.in
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274807#comment-13274807 ] Rick Hillegas commented on DERBY-5234: -- Hi Saurabh, The successful completion of the SELECT COUNT(*) is a good sign. At this point, I would try to dump the old, corrupted database and load it into a new 10.8.2.3 database. Thanks, -Rick > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapCo
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274800#comment-13274800 ] Saurabh Singhi commented on DERBY-5234: --- Hi Rick, The count query gives me 626677 rows. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown > Source)
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274776#comment-13274776 ] Rick Hillegas commented on DERBY-5234: -- Hi Saurabh, Hard to say where the OOM is coming from. It may be that the Squirrel Client runs out of memory while trying to prepare a big window for display. What happens if you SELECT COUNT(*) from the corrupted table? The VTI attached to DERBY-4962 will not try to materialize all of the results at once. You may be able to use it to stream the corrupted table a new, empty database. Thanks, -Rick > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.da
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274734#comment-13274734 ] Saurabh Singhi commented on DERBY-5234: --- Hi Rick, When I try to do a Select * on the table (via Squirrel Client) that I suspect is corrupted, it gives me a "java.lang.OutOfMemoryError: Java heap space" due to the fact that "The query results have exceeded the maximum amount of memory allocated to this application.". The allocated memory seems to be 247 MB. Could that be a cause for the fix not working? Thanks, Saurabh > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274690#comment-13274690 ] Rick Hillegas commented on DERBY-5234: -- Hi Saurabh, There is no additional fix. You just need a build from the 10.8 branch after revision 1337258. Thanks, -Rick > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.s
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274687#comment-13274687 ] Saurabh Singhi commented on DERBY-5234: --- Hi Rick, Should I get the latest branch again? Would it have this fix? > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLoca
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274684#comment-13274684 ] Rick Hillegas commented on DERBY-5234: -- Hi Saurabh, Can you SELECT all of the rows in the corrupted table? I find that the table can still be read after the corruption when I run my toy test case Derby5234. If you can SELECT all of the rows from the corrupted table, then I suggest the following course of action: A) Using 10.8.2.3, create a new database. B) Copy all of the data from the corrupted database into the new database. You can use the dump and bulk load procedures to do this. Alternatively, you can use the schema-importing VTI attached to DERBY-4962. The latter approach may be faster. C) Delete the old, corrupted database or rename it to make it clear that it should not be used in production anymore. D) Rename the new database to be the old name of the corrupted database. E) Continue running your application with 10.8.2.3 and the new (now renamed) database. Hope this helps, -Rick > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > Derby5234.java, derby-5234-01-aa-emptyAllocPage.diff, > derby-5234-01-ab-emptyAllocPage.diff, derby-5234-02-aa-edgeCaseTests.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.i
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273561#comment-13273561 ] Saurabh Singhi commented on DERBY-5234: --- Hi Rick, It is an existing/corrupted data table that I ran the fix against. It seems like after a certain limit (~1.3 gb), the database starts acting fuzzy and gets corrupted. Our application continually inserts data into the tables and periodically (once a day) deletes rows from the previous day and after the delete command, I am using the compress call to squeeze the db back into shape. Originally, I used the 'SYSCS_UTIL.SYSCS_COMPRESS_TABLE' call, but that lead to: DERBY-4275 & DERBY-5351. Following instructions, I used the 'INPLACE' call and was able to get rid of those 2 problems but now I'm stuck with this one. Thanks, Best, Saurabh > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > derby-5234-01-aa-emptyAllocPage.diff, derby-5234-01-ab-emptyAllocPage.diff, > derby-5234-02-aa-edgeCaseTests.diff, log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source)
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273549#comment-13273549 ] Rick Hillegas commented on DERBY-5234: -- Hi Saurabh, I just want to confirm whether your latest bug report was against a table which had been newly created with 10.8.2.3. Note the following: 1) I observe that I can still reboot a database which was already corrupted by the original form of this bug. And I can successfully delete all rows in the corrupted table and I can run inplace compress on the table. 2) The patch does not fix an already corrupted table. It only prevents a previously uncorrupted table from being corrupted this way. So I just want to confirm: Did you newly recreate the table using 10.8.2.3? Or was the table already corrupted and you were hoping that the patch would fix the broken table? Thanks, -Rick > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Assignee: Rick Hillegas >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Fix For: 10.8.2.3, 10.9.0.0 > > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > derby-5234-01-aa-emptyAllocPage.diff, derby-5234-01-ab-emptyAllocPage.diff, > derby-5234-02-aa-edgeCaseTests.diff, log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.ca
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[
https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273496#comment-13273496
]
Saurabh Singhi commented on DERBY-5234:
---
Hi Rick,
I built version 10.8.2.3, but unfortunately, I'm still getting the same errors.
Here's a dump of the same (note that the container exception occurs a little
deeper down the stack trace) :-
*
1 May 2012 11:29 DataDictionary.deleteHRData = deleteCmd = DELETE FROM
APP.CPU_059222_HR WHERE SampleTimestamp <= ?
11 May 2012 11:29 DataDictionary.deleteHRData = Delete executed with Timestamp
= Thu May 10 11:29:09 PDT 2012
11 May 2012 11:29 DataDictionary.reclaimDiskSpace = compressCmd:->call
SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE('APP', 'CPU_059222_HR',0,0, 1)
11 May 2012 11:29 DataDictionary.reclaimDiskSpace = compressSqlStmt executed
for Table:->CPU_059222_HR
11 May 2012 11:29 DataDictionary.deleteHRData = Hourly retention = 24
11 May 2012 11:29 DataDictionary.deleteHRData = deleteCmd = DELETE FROM
APP.PROCESS_059222_HR WHERE SampleTimestamp <= ?
11 May 2012 11:29 DataDictionary.deleteHRData = deleteHRData: SQL error - Java
exception: ': java.lang.NullPointerException'.
11 May 2012 11:29 SchemaDAO.tableExists = Java exception: ':
java.lang.NullPointerException'.
java.sql.SQLException: Java exception: ': java.lang.NullPointerException'.
at
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:98)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Util.java:142)
at org.apache.derby.impl.jdbc.Util.javaException(Util.java:299)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:436)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(TransactionResourceImpl.java:353)
at
org.apache.derby.impl.jdbc.EmbedConnection.handleException(EmbedConnection.java:2290)
at
org.apache.derby.impl.jdbc.EmbedConnection.(EmbedConnection.java:634)
at
org.apache.derby.impl.jdbc.EmbedConnection30.(EmbedConnection30.java:73)
at
org.apache.derby.impl.jdbc.EmbedConnection40.(EmbedConnection40.java:51)
at
org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Driver40.java:70)
at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:248)
at
org.apache.derby.jdbc.AutoloadedDriver.connect(AutoloadedDriver.java:144)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at
com.idelji.operations.iasset.plugin.imon.dao.SchemaDAO.tableExists(SchemaDAO.java:59)
at
com.idelji.operations.iasset.plugin.imon.dao.DataDictionary.deleteHRData(DataDictionary.java:464)
at
com.idelji.operations.iasset.plugin.imon.service.CleanUpTask.run(CleanUpTask.java:54)
Caused by: java.sql.SQLException: Java exception: ':
java.lang.NullPointerException'.
at
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(SQLExceptionFactory.java:45)
at
org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(SQLExceptionFactory40.java:122)
at
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:71)
... 16 more
Caused by: java.lang.NullPointerException
at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(BaseDataFileFactory.java:659)
at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(BaseDataFileFactory.java:589)
at
org.apache.derby.impl.store.raw.xact.Xact.openContainer(Xact.java:1316)
at
org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.init(OpenConglomerate.java:910)
at org.apache.derby.impl.store.access.heap.Heap.open(Heap.java:689)
at
org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(RAMTransaction.java:476)
at
org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(RAMTransaction.java:1308)
at
org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaIndexMinion(DataDictionaryImpl.java:9271)
at
org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaIndex(DataDictionaryImpl.java:9229)
at
org.apache.derby.impl.sql.catalog.DataDictionaryImpl.locateSchemaRow(DataDictionaryImpl.java:1722)
at
org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getSchemaDescriptor(DataDictionaryImpl.java:1589)
at
org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initDefaultSchemaDescriptor(GenericLanguageConnectionContext.java:425)
at
org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.ini
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273404#comment-13273404 ] Rick Hillegas commented on DERBY-5234: -- Hi Saurabh, The 10.9 development trunk contains what we think is a fix for this bug. I have just ported that fix to the 10.8 branch. The fix should appear in the 10.9 feature release which we will evaluate later this month: http://wiki.apache.org/db-derby/DerbyTenNineOneRelease There is also talk about producing a 10.8.3 maintenance release after the community publishes a 10.9 feature release. That maintenance release will also contain the fix. Hope this helps, -Rick > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > derby-5234-01-aa-emptyAllocPage.diff, derby-5234-01-ab-emptyAllocPage.diff, > derby-5234-02-aa-edgeCaseTests.diff, log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(U
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273397#comment-13273397 ] Rick Hillegas commented on DERBY-5234: -- Ported 1335570 and 1335677 from trunk to 10.8 branch at subversion revision 1337258. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > derby-5234-01-aa-emptyAllocPage.diff, derby-5234-01-ab-emptyAllocPage.diff, > derby-5234-02-aa-edgeCaseTests.diff, log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown > Source) > at org.apache.derby.impl.sql.execute.RowChangerImpl.in
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273382#comment-13273382 ] Saurabh Singhi commented on DERBY-5234: --- I am getting the same error, even after calling SYSCS_UTIL.SYSCS_COMPRESS_TABLE (or SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE ) programatically, before and/or after the DELETE statement. The DB size is 1.3 gb. Any updates on when a quick-fix or workaround or the actual fix would be available? This is an urgent requirement. Thanks. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > derby-5234-01-aa-emptyAllocPage.diff, derby-5234-01-ab-emptyAllocPage.diff, > derby-5234-02-aa-edgeCaseTests.diff, log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.a
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270522#comment-13270522 ] Rick Hillegas commented on DERBY-5234: -- Tests passed cleanly for me. Committed derby-5234-01-ab-emptyAllocPage.diff at subversion revision 1335570. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, Derby5234.java, > derby-5234-01-aa-emptyAllocPage.diff, derby-5234-01-ab-emptyAllocPage.diff, > log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown > Source) > at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknow
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268547#comment-13268547 ] Rick Hillegas commented on DERBY-5234: -- Thanks, Mike. I will add a test case to the patch. Please add comments on top of the patch once I've checked it in--I think that your comments will be more useful than mine. I will also put some thought into writing a follow-on patch containing tests for edge cases. Thanks. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, > derby-5234-01-aa-emptyAllocPage.diff, log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAn
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268498#comment-13268498 ] Mike Matrigali commented on DERBY-5234: --- the patch changes look right to me. the existing code (not just your changes) could use more comments, which probably led to the bugs in the first place. I like the changes to var names to make it clear they are bit positions of the pages not page numbers. I would be happy to add comments either after you checkin or produce a new patch based on your work, let me know how you want to procede. Definitely would be good to get a test case checked in with the patch. longer term - does not have to be part of this fix, trying to think of some complete edge case test, as at least fix for 1 is an off by one error. probably not for running every night. Something like 2k page heap, for (i up to some number bigger than a few extents) drop/create table each time add rows such that it fills i pages delete all rows inplace compress some set of tests, at least insert i + n rows > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, > derby-5234-01-aa-emptyAllocPage.diff, log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.de
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267007#comment-13267007 ] Mike Matrigali commented on DERBY-5234: --- Concerning the 3rd point above I have logged DERBY-5738. looks like intent was to handle multiple extents but at least in 10.1 and later the code does not handle shrinking past an extent. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, > derby-5234-01-aa-emptyAllocPage.diff, log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown > Source) > at org.apache.derby.impl.sql.execute.Row
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265148#comment-13265148 ] Rick Hillegas commented on DERBY-5234: -- Might be a red herring, but I see that there is only one ChainAllocPageOperation in the transaction logs. That occurs in log85.dat. It allocates page 10217 (2 pages before the bad page) and chains it to page 0. I would expect to see a second ChainAllocPageOperation after the compress. But I don't. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Attachments: 5234_alloc.out, 5234_page_10219.out, 5234_summary.out, > DataFileReader_Output.zip, DbCompressErrorTester.java, log191.dat, log85.dat > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(U
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262776#comment-13262776 ] Stefan Sitte commented on DERBY-5234: - @Mike: How can i set the flags for transaction logging ? > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Attachments: DataFileReader_Output.zip, DbCompressErrorTester.java > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown > Source) > at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown > Source) > at > org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown > Source) > at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown > Source) > at > org.apache.derb
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261022#comment-13261022 ] Mike Matrigali commented on DERBY-5234: --- What I would do if looking at this would be to see if you can get it to reproduce with flags set to keep all the transaction logs around. Using that with identifying when the page goes bad should get closer to where the problem is. I would look for correlation of when page alloc or page dealloc happens and when log records happen for the bad page. The stack printed has the feel of maybe looking for a page that is past the end of file? Putting some debugging about the page table and the end of file at that stack might help. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Attachments: DataFileReader_Output.zip, DbCompressErrorTester.java > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.im
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13260882#comment-13260882 ] Rick Hillegas commented on DERBY-5234: -- I also can reproduce this problem by running the repro which Stefan supplied. I will look into this bug. Hi Stefan: From the stack trace it appears that a base table has become corrupted. I don't know of any way to rebound from this other than to restore your database from the last good backup. Hopefully we will have a fix for this problem in 10.9, but that will only prevent the problem from happening again. The fix will not repair an already broken database. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Attachments: DataFileReader_Output.zip, DbCompressErrorTester.java > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) >
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13260654#comment-13260654 ] Stefan Sitte commented on DERBY-5234: - Is there any fix or workaround available ? It seems to be a problem of the cache. How can i rebuild the cache after the exception occurs ? > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: Store >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R >Priority: Critical > Labels: ERROR, XSDG0, apache, corruption, data, derby, > derby_triage10_9 > Attachments: DataFileReader_Output.zip, DbCompressErrorTester.java > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown > Source) > at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown > Source) > at > org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown > Source) > at org.apache.derby.impl
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165910#comment-13165910 ] Stefan Sitte commented on DERBY-5234: - That would be nice. I was suprised too, it could be reproduced in such an easy way. I hope it is a small thing to fix it. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > Attachments: DataFileReader_Output.zip, DbCompressErrorTester.java > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown > Source) > at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown > Source) > at > org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown > Source) > at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown > Source
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165711#comment-13165711 ] Dag H. Wanvik commented on DERBY-5234: -- Thanks for the enclosed repro, Stefan! It will increase your chances of getting a fix significantly :) I managed to reproduced the problem on Derby trunk. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > Attachments: DataFileReader_Output.zip, DbCompressErrorTester.java > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown > Source) > at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown > Source) > at > org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown > Source) > at org.apache.derby.impl.sql.execute.Ins
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039583#comment-13039583 ] Varma R commented on DERBY-5234: @Dag - Inplace Compress command is called once every day. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > Attachments: DataFileReader_Output.zip > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown > Source) > at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown > Source) > at > org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown > Source) > at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown > Source) > at > org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source) > at org.apac
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039382#comment-13039382 ] Mike Matrigali commented on DERBY-5234: --- it is interesting that the case is exactly at a new extent which may or may not be the issue. It you have any other examples of this problem it would help to know if the end of the file looks the same as this one. Again once this problem happens a number of things are likely to break, and the real problem is not the insert but what allowed the allocate of the free pages to finish without the space on disk being forced. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > Attachments: DataFileReader_Output.zip > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown > Source) >
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039377#comment-13039377 ] Mike Matrigali commented on DERBY-5234: --- I am not surprised that SYSCS_UTIL.SYSCS_COMPRESS_TABLE fixed the problem, as it seems the issue is in just the last few pages of the file that are marked free but not actually on disk. This version of compress which I refer to as offline, just reads through the used pages of the file and inserts into the new table and never looks at the problem empty and not on disk pages at the end of the file. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > Attachments: DataFileReader_Output.zip > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown > Source) > at org.apache.derby.i
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039363#comment-13039363 ] Mike Matrigali commented on DERBY-5234: --- The issues here all have the feel of the database getting broken sometime in past. Usually once broken not much can be done, including upgrading - other than rebuild. I would suggest running the consistency checker across the whole database to see if there are other hidden problems lurking. Given the end of file issues it would be interesting to know: o Do you run online or offline compress on this table (before being advised for this issue). o I am not too familar with HP unix, so not sure if these questions are applicable. Can you describe the file system that derby is running on there. Is there any chance that I/O system forces through java are not being respected by the filesystem/hardware. On other unix systems there are options available to force wait on synchronous writes, and derby depends on this. Unfortunately often the default is to pretend that write sync is happening when it is not and this can cause problems for Derby. Without the actual db it is hard to know what is going on exactly from just the stack. I thought derby was meant to be able to handle some "missing" pages at the end of the file, so this would be one case where it might be useful to try a more recent version of the Derby software. It could be that I am remembering recovery logic rather than runtime logic. The missing pages feels like a lost "force" of the free pages thus the questions about filesystem/hardware settings. Running the consistency checker could find other lsot forces that may have existed. This kind of error can also come from users manipulating files in the log directory. Have you ever "fixed" a problem where the database would not boot by removing files from the log directory? In trying to understand what might have happened to cause the bug, can you correlate at all the problems happening with machine crashes at all? > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > Attachments: DataFileReader_Output.zip > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij>
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038555#comment-13038555 ] Rick Hillegas commented on DERBY-5234: -- Hi, Dag. By default, the DataFileReader tool reads until it reaches EOF. You can also tell it to stop reading after a fixed number of pages. The page size of c450.dat is 4096, the file is 212660224 bytes long according to Varma's comment on 2011-05-18, so the number of pages in the file is 212660224 / 4096 = 51919. This agrees with what the tool found: the tool successfully decoded pages 0 - 51918. The extent lengths are: grep extentLength Output_c450.dat.xml 10216 10424 10424 10424 10424 24 It does seem odd that the last allocation page thinks there are 24 more pages but there is really only 1 more page. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > Attachments: DataFileReader_Output.zip > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseCo
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038293#comment-13038293 ] Dag H. Wanvik commented on DERBY-5234: -- The dump of the data pages seem to indicate that the file has no pages from page 51919 onwards (the last dumped is 515918 on c450.dat). The data dumped show that page 51917 has an extent that coincides with the one we saw dumped: Start/end:: 51918:62341. And page 51918 is indeed present and valid in the dump. But no further pages are printed. Rick, does the tool continue printiing till it reaches EOF or does it stop according some variable in the container stating how many pages are allocated? I am not very familiar with this code at all, but doesn't the presence of the extent information on page 51917 indicated that pages from 51918:62341 should be allocated (e.g be not handled back to the operating system after an inline compress with truncate end option[1])? [1] http://db.apache.org/derby/docs/10.5/ref/rrefproceduresinplacecompress.html Btw, Varma, does the app ever perform in-place compress? > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > Attachments: DataFileReader_Output.zip > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.a
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038147#comment-13038147 ] Rick Hillegas commented on DERBY-5234: -- Hi Varma, It's hard to say if an upgrade will fix this problem since we don't understand the root cause yet. The reason I asked whether your application experiences interrupts is that they could potentially result in a data corruption on version 10.5.3.0. Substantial work was put into protecting Derby from interrupts in 10.8.1.2. Can you say whether your application might be experiencing thread interrupts? Thanks, -Rick > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > Attachments: DataFileReader_Output.zip > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown > Source) > at
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13037748#comment-13037748 ] Varma R commented on DERBY-5234: Current Version of Derby been used in production is 10.5.3.0. Will the problem be resolved by upgrading derby database to latest version(which is 10.8.1.2)? > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > Attachments: DataFileReader_Output.zip > > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown > Source) > at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown > Source) > at > org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown > Source) > at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown > Source) > at > o
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13035551#comment-13035551 ] Rick Hillegas commented on DERBY-5234: -- Hi Varma, This may be a red herring, but it is worth asking: does your application experience thread interrupts? Some useful information might surface if you run the DataFileReader tool against the corrupt heap conglomerate. The tool is attached to DERBY-5201. Redirect the output to a file and then attach that file to this JIRA issue. Help on the tool's usage can be obtained by typing java DataFileReader I think that the following experiment would be useful (where $dbName is the name of your database directory): java DataFileReader $dbName/seg0/c450.dat -v -p 51916 -n 6 > c450_partial.xml You can also use the -d option of the tool if you are willing to disclose the data which appears in the closing pages of the file. See the help output for a description of how to use the -d option. You can, of course, redact any sensitive information in the output. Thanks, -Rick > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) >
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13035420#comment-13035420 ] Varma R commented on DERBY-5234: @Bryan, I doubt. The corrupted DB was moved from Production (HP-UX) to test environment (Solaris & Windows) and the DB was started with the exact settings. All three environments return the same error on INSERT in the KPI.KPI_MERGE_IN table > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown > Source) > at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown > Source) > at > org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown > Source) > at org.apache.derby.impl.sql.execute.InsertResultSet.open
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13035388#comment-13035388 ] Bryan Pendleton commented on DERBY-5234: Any chance you're hitting a filesystem quota of some sort? Have you checked all the ulimit-like settings? This feels like the sort of problem where the operating system is truncating the file or partially writing it because the operating system thinks the process has hit its limit. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown > Source) > at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown > Source) > at > org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown > Source) >
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13035384#comment-13035384 ] Varma R commented on DERBY-5234: @Kristian, Thanks for the help. The data however can't be made public 1) Ran the corrupt db data with debug libraries. Here is the output in derby.log ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:336) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:695) at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPage.java:190) at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:295) at org.apache.derby.impl.store.raw.data.FileContainer.initPage(FileContainer.java:2317) at org.apache.derby.impl.store.raw.data.FileContainer.newPage(FileContainer.java:1800) at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(BaseContainer.java:314) at org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(BaseContainerHandle.java:183) at org.apache.derby.impl.store.access.heap.HeapController.doInsert(HeapController.java:302) at org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(HeapController.java:599) at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(RowChangerImpl.java:452) at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(InsertResultSet.java:1022) at org.apache.derby.impl.sql.execute.InsertResultSet.open(InsertResultSet.java:495) at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(GenericPreparedStatement.java:416) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:297) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1235) at org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:625) at org.apache.derby.impl.jdbc.EmbedStatement.executeUpdate(EmbedStatement.java:175) at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLIMM(DRDAConnThread.java:5022) at org.apache.derby.impl.drda.DRDAConnThread.processCommands(DRDAConnThread.java:750) at org.apache.derby.impl.drda.DRDAConnThread.run(DRDAConnThread.java:290) Caused by: java.io.EOFException: Reached end of file while attempting to read a whole page. at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContainer4.java:474) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage0(RAFContainer4.java:242) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContainer4.java:212) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:671) ... 19 more = begin nested exception, level (1) === java.io.EOFException: Reached end of file while attempting to read a whole page. at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContainer4.java:474) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage0(RAFContainer4.java:242) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContainer4.java:212) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:671) at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPage.java:190) at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:295) at org.apache.derby.impl.store.raw.data.FileContainer.initPage(FileContainer.java:2317) at org.apache.derby.impl.store.raw.data.FileContainer.newPage(FileContainer.java:1800) at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(BaseContainer.java:314) at org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(BaseContainerHandle.java:183) at org.apache.derby.impl.store.access.heap.HeapController.doInsert(HeapController.java:302) at org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(HeapController.java:599) at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(RowChangerImpl.java:452) at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(InsertResultSet.java:1022) at org.apache.derby.impl.sql.execute.InsertResultSet.open(InsertResultSet.java:495) at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(GenericPreparedStatement.java:416) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:297) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1235) at org.apache.derby.impl.jdbc.EmbedStatement.ex
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13035242#comment-13035242 ] Kristian Waagan commented on DERBY-5234: Varma, some comments below: a) It's not clear to me if you run with the Derby debug build, i.e. the debug bundle named "db-derby-10.x.y.z-lib-debug.zip" (downloadable from the Apache Derby site). My hope was that it would detect if there was invalid metadata when accessing the table. b) This is probably a dead end anyway (unless item d suggests otherwise). In any case, I think it would only matter on the production system since I assume that's where the data was actually written to disk. c) Interesting. So it appears that the compress operation is able to rectify the state that's causing the insert to fail. If asked for, could the data file for the apparently corrupted table (or the whole database) be made public? Compressing a table is done to reclaim unused disk space for the base table and the indexes. If your load consists of a high rate of deletes and updates, it might be worth running it periodically if possible. See the Reference Guide and "Reclaiming unused space" in the Derby Server and Administration Guide for details. I'm hoping someone with a more intimate knowledge about the Derby store can point us in the right direction, and it would be nice to obtain the size of the corrupted table. It seems Derby fails to read back a newly created page, which is odd... > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.i
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13034987#comment-13034987 ] Rick Hillegas commented on DERBY-5234: -- Hi Varma, To find the data files corresponding to your tables, you can run the following script: create function toHexString( a bigint ) returns varchar( 25 ) language java parameter style java no sql external name 'java.lang.Long.toHexString'; select t.tableName, 'c' || toHexString( c.conglomerateNumber ) || '.dat' from sys.systables t, sys.sysconglomerates c, sys.sysschemas s where t.tableID = c.tableID and t.schemaID = s.schemaID and s.schemaName not like 'SYS%'; drop function toHexString; > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source) > at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown > Source) > at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown > Source) > at > org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown >
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13034679#comment-13034679 ] Varma R commented on DERBY-5234: Hi, Tried the above options mentioned. a) Given the below debug option which starting the derby database. -Dderby.debug.true=CacheTrace,DaemonTrace,DeadLockTrace,LockStackTrace,LockTrace,LogTrace,memoryLeakTrace,ScanTrace,SerializedTrace,SpaceTrace,TEST_LOG_FULL But this didn't print any additional information in the logs. b) Below are the command output's on test environment. -bash-3.00$ ulimit -a | grep file core file size(blocks, -c) unlimited file size (blocks, -f) unlimited open files(-n) 256 -bash-3.00$ plimit 5036 5036: java -Dderby.system.home=/export/home/cmd -Dderby.locks.monitor=true - resource current maximum time(seconds) unlimited unlimited file(blocks) unlimited unlimited data(kbytes) unlimited unlimited stack(kbytes) 10240 unlimited coredump(blocks) unlimited unlimited nofiles(descriptors) 65536 65536 vmemory(kbytes) unlimited unlimited you can see that there are no limits on the file system and 65536 is the hard limit for the file descriptors. c) After running SYSCS_UTIL.SYSCS_COMPRESS_TABLE procedure on KPI.KPI_MERGEIN table. INSERT command is working on KPI.KPI_MERGEIN table. d) How to find out the list of .dat files corresponding to a table in seg0 directory? This same application was load tested for higher transcation volume and no such error were reported during development. However the problem in production seems to be pointing to cumulative data/disk usage. Also would it be recommeneded to run SYSCS_UTIL.SYSCS_COMPRESS_TABLE once in a while? > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchit
[jira] [Commented] (DERBY-5234) Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
[ https://issues.apache.org/jira/browse/DERBY-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13034210#comment-13034210 ] Kristian Waagan commented on DERBY-5234: Hi, I can't say what has gone wrong, but have a few initial questions that may inform the debugging process: a) Can you run with the debug version of Derby? This would be best during normal operation, before the error has happened, but may not work because all the extra asserts slow down Derby a lot. Booting and accessing (SELECT/INSERT) the corrupted database with the debug version is also useful in some cases, and that's a lot simpler to do. b) Have you verified that there are no limits on the size of the database files, i.e. OS quotas or file system limitations, in the production environment? c) What happens if you run SYSCS_UTIL.SYSCS_COMPRESS_TABLE on KPI.KPI_MERGEIN after the insert fails? If the compress-operation succeeds, does the insert still fail afterwards? d) Looking at the data file for the corrupted table, is the size a multiple of the page size? The failing code is using NIO, but there doesn't seem to be a problem with interrupts (based on current info only). The two obvious ways the code can fail are if the file for some reason has been truncated or that the position argument is incorrect. > Unable to insert data into table. Failed due be "ERROR XSDG0: Page > Page(51919,Container(0, 1104)) could not be read from disk." > --- > > Key: DERBY-5234 > URL: https://issues.apache.org/jira/browse/DERBY-5234 > Project: Derby > Issue Type: Bug > Components: JDBC, Network Server, SQL >Affects Versions: 10.5.3.0 > Environment: HP-UX 11iv2 in production environment with JDK1.6; > Solaris 5/10 in test environment with JDK 1.6 >Reporter: Varma R > Labels: ERROR, XSDG0, apache, corruption, data, derby > > One of the derby database table "gets corrupted"/"indicates connection not > available" during processing inserts from java client application as shown in > the trace and the only way to recover from this error is to rebuild the DB - > by deleting the data and creating the tables again. This happens once in a > while (thrice in a span of two months) and the java application (run in > multiple servers), which updates the database, processes around 100 million > transactions per hour (in total and each transation results in 4-5 updates to > the DB) > There are eight tables in the derby database. >TABLE NAME ROWS COUNT (at time of corruption) > - >KPI.KPI_MERGEIN; 362917 >KPI.KPI_IN; 422508 >KPI.KPI_DROPPED;53667 >KPI.KPI_ERROR1; 0 >KPI.KPI_ERROR2; 2686 >KPI.KPI_ERRORMERGE;0 >KPI.KPI_MERGEOUT; 362669 >KPI.KPI_OUT; 125873 > The derby database has been started with the following parameters > CMD="java -Dderby.system.home=$DERBY_OPTS -Dderby.locks.monitor=true > -Dderby.locks.deadlockTrace=true -Dderby.locks.escalationThreshold=5 > -Dderby.locks.waitTimeout= > -1 -Dderby.storage.pageCacheSize=10 -Xms512M -Xmx3072M -XX:NewSize=256M > -classpath $DERBY_CLASSPATH org.apache.derby.drda.NetworkServerControl start > -h $KPIDERBYHOST -p $DERBY_KPI_PORT" > The corrupted database tar (filesystem) in live environment was moved to a > test system (Solaris system) and few checks were run on the corrupted DB as > part of analysis (DB does start fine) > While trying to insert a row in any table expect KPI.KPI_MERGEIN, it is > successful. But when a new row is inserted into KPI.KPI_MERGEIN table using > command line tool it's throwing below error message (the same message that > appeared in live > ij> INSERT INTO KPI.KPI_MERGEIN (A0_TXN_ID, A1_NE_ID, A2_CHU_IP_ADDR, > A3_BATCH_DATE,A5_CODE) VALUES (-1, 'BMTDE', '192.2.1.3', 231456879, 'KSD'); > ERROR 08006: A network protocol error was encountered and the connection has > been terminated: the requested command encountered an unarchitected and > implementation-specific condition for which there was no architected message > and in derby.log file it shows below error stacktrace. > ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk. > at org.apache.derby.iapi.error.StandardException.newException(Unknown > Source) > at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown > Source) > at > org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source) > at
