A production system with a database of about 400G received this error today and
it appears that Derby shutdown parts of itself because from that point on it
was spitting out errors saying it could not find the database. The system was
restarted and came up clean and is working with no issues inserting data at
well over 100 inserts/second continuous and performing queries and other data
updates with no issues.
ERROR XSDG0: Page Page(1325564,Container(0, 30832)) could not be read from disk.
at org.apache.derby.iapi.error.StandardException.newException(Unknown
Source)
at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown
Source)
at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown
Source)
at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown
Source)
at org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown
Source)
at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown
Source)
at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown
Source)
at
org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown Source)
at
org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown Source)
at
org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown
Source)
at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown
Source)
at
org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown
Source)
at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown
Source)
at
org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
Source)
at
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown
Source)
at org.apache.derby.iapi.jdbc.BrokeredPreparedStatement.execute(Unknown
Source)
at org.apache.derby.impl.drda.DRDAStatement.execute(Unknown Source)
at
org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTTobjects(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
Caused by: java.io.EOFException: Reached end of file while attempting to read a
whole page.
at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(Unknown
Source)
at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage0(Unknown
Source)
at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(Unknown
Source)
at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(Unknown
S:
derby the reported errors like:
Thu Sep 03 16:03:54 GMT 2015 Thread[DRDAConnThread_80,5,main] (DATABASE =
csemdb), (DRDAID = ????????.??-1011901099978582906{5640754}),
Thu Sep 03 16:03:54 GMT 2015 :
org.apache.derby.iapi.error.ShutdownException:
at
org.apache.derby.iapi.services.context.ContextManager.checkInterrupt(Unknown
Source)
at
org.apache.derby.iapi.services.context.ContextManager.popContext(Unknown Source)
at org.apache.derby.iapi.services.context.ContextImpl.popMe(Unknown
Source)
at org.apache.derby.jdbc.EmbedXAResource.removeXATransaction(Unknown
Source)
at
org.apache.derby.jdbc.EmbedXAResource.returnConnectionToResource(Unknown Source)
at org.apache.derby.jdbc.EmbedXAResource.commit(Unknown Source)
at
org.apache.derby.impl.drda.DRDAXAProtocol.commitXATransaction(Unknown Source)
at org.apache.derby.impl.drda.DRDAXAProtocol.commitTransaction(Unknown
Source)
at org.apache.derby.impl.drda.DRDAXAProtocol.parseSYNCCTL(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
Thu Sep 03 16:03:54 GMT 2015 Thread[DRDAConnThread_114,5,main] (DATABASE =
csemdb), (DRDAID = ????????.??-940969405847491181{5640766}), null
Thu Sep 03 16:03:54 GMT 2015 : null
Thu Sep 03 16:03:54 GMT 2015 : null
java.lang.NullPointerException
at org.apache.derby.impl.drda.DRDAConnThread.writePBSD(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
java.lang.NullPointerException
at org.apache.derby.impl.drda.DRDAConnThread.writePBSD(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
Thu Sep 03 16:03:54 GMT 2015 Thread[DRDAConnThread_191,5,main] (DATABASE =
csemdb), (DRDAID = ????????.??-939843505940648198{5640770}), null
Thu Sep 03 16:03:54 GMT 2015 : null
And
Thu Sep 03 16:03:54 GMT 2015 Thread[DRDAConnThread_185,5,main] Cleanup action
starting
java.sql.SQLException: Database 'csemdb' not found.
at
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.newSQLException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleDBNotFound(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)
at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.EmbeddedBaseDataSource.getConnection(Unknown
Source)
at
org.apache.derby.jdbc.EmbedPooledConnection.openRealConnection(Unknown Source)
at org.apache.derby.jdbc.EmbedXAConnection.getRealConnection(Unknown
Source)
at
org.apache.derby.iapi.jdbc.BrokeredConnection.getRealConnection(Unknown Source)
at org.apache.derby.iapi.jdbc.BrokeredConnection.isClosed(Unknown Source
The system is an Oracle M5000 Enterprise server with what I believe is a 15TB
Sun ZFS Storage 7320 external ZFS storage array connected by Fibre Channel.
This is the first time in over 8 years we have seen any I/O error like such.
What I am trying to confirm is that this is really low level derby code that if
it reports an "java.io.EOFException" like it did, it really did have an I/O
error somewhere in reading the page from the container file. Things like
performance, java heap space, etc, can pretty much be ruled out as causing such
an error. My gut feeling is that maybe something in the connection to this
storage array had a hiccup. This setup is at the customer site and I cannot
directly access system logs nor do I have knowledge on how this storage array
works and how to look at such but just having confirmation that an I/O error
really did occur would help.
________________________________
Canoga Perkins
20600 Prairie Street
Chatsworth, CA 91311
(818) 718-6300
This e-mail and any attached document(s) is confidential and is intended only
for the review of the party to whom it is addressed. If you have received this
transmission in error, please notify the sender immediately and discard the
original message and any attachment(s).
