Geronimo Derby database corrupted after unix server restart (geronimo running
at the time of restart)
-----------------------------------------------------------------------------------------------------
Key: GERONIMO-5785
URL: https://issues.apache.org/jira/browse/GERONIMO-5785
Project: Geronimo
Issue Type: Bug
Security Level: public (Regular issues)
Components: startup/shutdown
Affects Versions: 2.1.4
Reporter: Daniel Stoddart
Hi,
We have several process running an a unix server and if we are not aware that
it is going to be restarted and Geronimo is not shutdown first via the
shutdown.sh script or through the Console we cannot restart geronimo and need
to reinstall which can take some time.
This is what we see in /var/log/geronimo.log
2011-01-26 10:51:05,289 ERROR [GBeanInstanceState] Error while starting; GBean
is now in the FAILED state:
abstractName="org.apache.geronimo.configs/transaction/2.1.4/car?ServiceModule=org.apache.geronimo.configs/transaction/2.1.4/car,j2eeType=TransactionLog,name=HOWLTransactionLog"
org.objectweb.howl.log.LogConfigurationException: Unable to obtain lock on
/home/ceh1/poseidon_dev/janap01/apps/uat/uat.geronimo-tomcat6-javaee5-2.1.4/./var/txlog/howl_1.log
at org.objectweb.howl.log.LogFile.open(LogFile.java:191)
at org.objectweb.howl.log.LogFileManager.open(LogFileManager.java:784)
at org.objectweb.howl.log.Logger.open(Logger.java:303)
at org.objectweb.howl.log.xa.XALogger.open(XALogger.java:893)
at org.apache.geronimo.transaction.log.HOWLLog.doStart(HOWLLog.java:224)
at
org.apache.geronimo.gbean.runtime.GBeanInstance.createInstance(GBeanInstance.java:998)
at
org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:268)
at
org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:102)
at
org.apache.geronimo.gbean.runtime.GBeanInstance.start(GBeanInstance.java:541)
at
org.apache.geronimo.gbean.runtime.GBeanDependency.attemptFullStart(GBeanDependency.java:111)
at
org.apache.geronimo.gbean.runtime.GBeanDependency.addTarget(GBeanDependency.java:146)
at
org.apache.geronimo.gbean.runtime.GBeanDependency$1.running(GBeanDependency.java:120)
at
org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.fireRunningEvent(BasicLifecycleMonitor.java:176)
at
org.apache.geronimo.kernel.basic.BasicLifecycleMonitor.access$300(BasicLifecycleMonitor.java:44)
at
org.apache.geronimo.kernel.basic.BasicLifecycleMonitor$RawLifecycleBroadcaster.fireRunningEvent(BasicLifecycleMonitor.java:254)
at
org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:294)
at
org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:102)
at
org.apache.geronimo.gbean.runtime.GBeanInstanceState.startRecursive(GBeanInstanceState.java:124)
at
org.apache.geronimo.gbean.runtime.GBeanInstance.startRecursive(GBeanInstance.java:555)
at
org.apache.geronimo.kernel.basic.BasicKernel.startRecursiveGBean(BasicKernel.java:379)
at
org.apache.geronimo.kernel.config.ConfigurationUtil.startConfigurationGBeans(ConfigurationUtil.java:456)
at
org.apache.geronimo.kernel.config.KernelConfigurationManager.start(KernelConfigurationManager.java:188)
at
org.apache.geronimo.kernel.config.SimpleConfigurationManager.startConfiguration(SimpleConfigurationManager.java:563)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.geronimo.gbean.runtime.ReflectionMethodInvoker.invoke(ReflectionMethodInvoker.java:34)
at
org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:124)
at
org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:832)
at
org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57)
at
org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35)
at
org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96)
at
org.apache.geronimo.gbean.GBeanLifecycle$$EnhancerByCGLIB$$dd83360e.startConfiguration(<generated>)
at
org.apache.geronimo.system.main.EmbeddedDaemon.doStartup(EmbeddedDaemon.java:162)
at
org.apache.geronimo.system.main.EmbeddedDaemon.execute(EmbeddedDaemon.java:79)
at
org.apache.geronimo.kernel.util.MainConfigurationBootstrapper.main(MainConfigurationBootstrapper.java:45)
at org.apache.geronimo.cli.AbstractCLI.executeMain(AbstractCLI.java:67)
at org.apache.geronimo.cli.daemon.DaemonCLI.main(DaemonCLI.java:30)
If we delete/reinstate the how1_1.log file we then get to another error which
seems to be the real problem:
2011-01-26 11:13:29,843 ERROR [MCFConnectionInterceptor] Error occurred
creating ManagedConnection for
org.apache.geronimo.connector.outbound.ConnectionInfo@18dd7403
javax.resource.spi.ResourceAdapterInternalException: Unable to obtain physical
connection to org.apache.derby.jdbc.EmbeddedXADataSource@27daf835
at
org.tranql.connector.jdbc.AbstractXADataSourceMCF.getPhysicalConnection(AbstractXADataSourceMCF.java:76)
at
org.tranql.connector.derby.EmbeddedXAMCF.createManagedConnection(EmbeddedXAMCF.java:53)
at
org.apache.geronimo.connector.outbound.MCFConnectionInterceptor.getConnection(MCFConnectionInterceptor.java:49)
at
org.apache.geronimo.connector.outbound.XAResourceInsertionInterceptor.getConnection(XAResourceInsertionInterceptor.java:41)
at
org.apache.geronimo.connector.outbound.SinglePoolConnectionInterceptor.internalGetConnection(SinglePoolConnectionInterceptor.java:71)
at
org.apache.geronimo.connector.outbound.AbstractSinglePoolConnectionInterceptor.getConnection(AbstractSinglePoolConnectionInterceptor.java:80)
at
org.apache.geronimo.connector.outbound.TransactionEnlistingInterceptor.getConnection(TransactionEnlistingInterceptor.java:46)
at
org.apache.geronimo.connector.outbound.TransactionCachingInterceptor.getConnection(TransactionCachingInterceptor.java:96)
at
org.apache.geronimo.connector.outbound.TCCLInterceptor.getConnection(TCCLInterceptor.java:39)
at
org.apache.geronimo.connector.outbound.AbstractConnectionManager.doRecovery(AbstractConnectionManager.java:65)
at
org.apache.geronimo.connector.outbound.ManagedConnectionFactoryWrapper.doStart(ManagedConnectionFactoryWrapper.java:166)
at
org.apache.geronimo.gbean.runtime.GBeanInstance.createInstance(GBeanInstance.java:998)
at
org.apache.geronimo.gbean.runtime.GBeanInstanceState.attemptFullStart(GBeanInstanceState.java:268)
at
org.apache.geronimo.gbean.runtime.GBeanInstanceState.start(GBeanInstanceState.java:102)
.......
Caused by: java.sql.SQLException: Failed to start database 'SystemDatabase',
see the next exception for details.
at
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)
at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(Unknown
Source)
at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(Unknown
Source)
at
org.apache.derby.jdbc.EmbeddedXADataSource.setupResourceAdapter(Unknown Source)
at org.apache.derby.jdbc.EmbeddedXADataSource.getXAConnection(Unknown
Source)
at
org.tranql.connector.jdbc.AbstractXADataSourceMCF.getPhysicalConnection(AbstractXADataSourceMCF.java:74)
... 44 more
Caused by: java.sql.SQLException: Failed to start database 'SystemDatabase',
see the next exception for details.
at
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at
org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
Source)
... 58 more
Caused by: java.sql.SQLException: Another instance of Derby may have already
booted the database
/home/ceh1/poseidon_dev/janap01/apps/uat/uat.geronimo-tomcat6-javaee5-2.1.4/var/derby/SystemDatabase.
at
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at
org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
Source)
at
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
Source)
... 55 more
Caused by: ERROR XSDB6: Another instance of Derby may have already booted the
database
/home/ceh1/poseidon_dev/janap01/apps/uat/uat.geronimo-tomcat6-javaee5-2.1.4/var/derby/SystemDatabase.
at org.apache.derby.iapi.error.StandardException.newException(Unknown
Source)
at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown
Source)
at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown
Source)
at java.security.AccessController.doPrivileged(Native Method)
at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown
Source)
at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source)
at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown
Source)
at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown
Source)
at
org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
at
org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown
Source)
at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown
Source)
at
org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
at org.apache.derby.iapi.services.monitor.Moni......
Is there a clean up script available or instructions on how to reinstall Derby
without re-installing the entire geronimo server? Any help would be appreciated
as I spend most of a day sorting this out once a month to re-instate our dev
and uat geronimo instances.
Thanks
Dan
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
