files starting with d are most usually associated with table, indexes,
and/or
constraints that have been dropped. In this case derby renames the
original cXXX.dat file to dXXX.dat file, and "later" will eventually
delete the dXXX.dat file. It does this so that if the drop is part
of a transaction that aborts it can recover from the drop and rename
the original file back.
By "later" I think it is is the next checkpoint for the system where
where the transaction that did the drop has been committed or aborted.
There are a number of other operations which internally look like
a "drop" to the storage system. I believe they include:
off line compress
some set of add column calls
some set of change column type/length calls
not sure about drop column
What you are describing seems likely to be a bug in
SYSCS_UTIL.SYSCS_BACKUP_DATABASE_NOWAIT(). It seems like this routine
either should be disallowing the d files to go away if the backup really
needs it, or needs to handle
it disappearing if the backup does not actually need it. I thought it
tried to make ddl type operations block
during backup, but not sure. It could be an edge case that is not
handled, if you can write a repro test case, please log a JIRA and
include the test case. I wonder if the issue is a dropped file that
has been committed before the backup, but has not yet been cleaned up
and then gets cleaned up during the backup?
On 3/11/2013 10:20 AM, Brandon L. Duncan wrote:
This is more of a general Store question precipitated by a specific
error I've come across during execution of the backup procedure.
Typically, I know that the seg0 folder maintains a file per conglomerate
in the format of cXXX.dat where XXX is the hex value of the conglomerate
id (Thank you Mr. Pendleton's blog). In what case would Derby expect or
create a .dat file starting with 'd'?
The instance of Derby in question is running fine under normal
operation, however when running a backup, the procedure fails due to a
missing .dat file that happens to being with 'd' (exception posted
below.). Is there any way to tell from the system tables, why derby
believes this file should be there, or more specifically, what derby
thinks should be stored in this file? Is it another conglomerate that is
perhaps corrupted?
This is from 10.8.3.0.
Mon Mar 11 12:59:26 EDT 2013 Thread[DRDAConnThread_5,5,main] (XID =
4666255288), (SESSIONID = 1), (DATABASE = db), (DRDAID =
????????.??-593347887885188381{1}), Cleanup action starting
Mon Mar 11 12:59:26 EDT 2013 Thread[DRDAConnThread_5,5,main] (XID =
4666255288), (SESSIONID = 1), (DATABASE = db), (DRDAID =
????????.??-593347887885188381{1}), Failed Statement is: CALL
SYSCS_UTIL.SYSCS_BACKUP_DATABASE_NOWAIT('./backup')
ERROR XSRS5: Error copying file (during backup) from
/derby/db/seg0/D2f410.DAT to ./backup/db/seg0/D2f410.DAT.
at
org.apache.derby.iapi.error.StandardException.newException(StandardException.java:303)
at
org.apache.derby.impl.store.raw.data.RAFContainer.privBackupContainer(RAFContainer.java:1071)
at
org.apache.derby.impl.store.raw.data.RAFContainer.run(RAFContainer.java:1650)
at java.security.AccessController.doPrivileged(Native Method)
at
org.apache.derby.impl.store.raw.data.RAFContainer.backupContainer(RAFContainer.java:980)
at
org.apache.derby.impl.store.raw.data.BaseContainerHandle.backupContainer(BaseContainerHandle.java:1031)
at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.backupDataFiles(BaseDataFileFactory.java:2551)
at org.apache.derby.impl.store.raw.RawStore.backup(RawStore.java:944)
at org.apache.derby.impl.store.raw.RawStore.backup(RawStore.java:653)
at
org.apache.derby.impl.store.access.RAMAccessManager.backup(RAMAccessManager.java:964)
at org.apache.derby.impl.db.BasicDatabase.backup(BasicDatabase.java:419)
at
org.apache.derby.catalog.SystemProcedures.SYSCS_BACKUP_DATABASE_NOWAIT(SystemProcedures.java:875)
at
org.apache.derby.exe.acf81e0010x013dx5a5fxee17x0000105f54e52.g0(Unknown
Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.derby.impl.services.reflect.ReflectMethod.invoke(ReflectMethod.java:46)
at
org.apache.derby.impl.sql.execute.CallStatementResultSet.open(CallStatementResultSet.java:75)
at
org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(GenericPreparedStatement.java:438)
at
org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:319)
at
org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1242)
at
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(EmbedPreparedStatement.java:1692)
at
org.apache.derby.impl.jdbc.EmbedCallableStatement.executeStatement(EmbedCallableStatement.java:117)
at
org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(EmbedPreparedStatement.java:1347)
at org.apache.derby.impl.drda.DRDAStatement.execute(DRDAStatement.java:706)
at
org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(DRDAConnThread.java:4164)
at
org.apache.derby.impl.drda.DRDAConnThread.processCommands(DRDAConnThread.java:1037)
at org.apache.derby.impl.drda.DRDAConnThread.run(DRDAConnThread.java:295)
Cleanup action completed
