I know for a fact that the issue is our NFS mount. We have experienced it with other tools, and those tools provided options for locking via other mechanisms. I've attempted to set "?broker.persistent=false" in all of my connections (I'm not explicitly starting any brokers in my application as I have external ActiveMQ brokers running), so I'm not sure what VM instance could be trying to create a persistent journal. In the one VM connection we had, I changed it to be TCP and it still was attempting to start a VM broker and failed with the same error. I'm pretty much at a loss at the moment.
-Ryan -----Original Message----- From: Stan Lewis [mailto:[email protected]] Sent: Tuesday, January 18, 2011 5:15 PM To: [email protected] Subject: Re: ActiveMQ and NFS mounts It looks like you're running out of file locks at the operating system level. You'll have to do a bit of research to see what the default resource limit is for the max number of file locks allowed on your operating system and see how to change it. This can happen whether or not you're running with the store on an NFS mount, it tends to be more prevalent with NFS though because NFS doesn't always support file locks, or typically the server process that manages distributed locks isn't running. In this case though you're hitting a kernel resource limit for file locks. Or worst case you could probably switch to an in-memory store for your CI builds and avoid the issue altogether. On Tue, Jan 18, 2011 at 6:14 PM, Shelley, Ryan <[email protected]> wrote: > Here's the actual stacktrace: > > 2011-01-18 04:36:25,488 [main] ERROR org.apache.activemq.broker.BrokerService > - Failed to start ActiveMQ JMS Message Broker. Reason: java.io.IOException: > No locks available > java.io.IOException: No locks available > at sun.nio.ch.FileChannelImpl.lock0(Native Method) > at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:881) > at java.nio.channels.FileChannel.tryLock(FileChannel.java:962) > at > org.apache.activemq.store.amq.AMQPersistenceAdapter.doLock(AMQPersistenceAdapter.java:948) > at > org.apache.activemq.store.amq.AMQPersistenceAdapter.lock(AMQPersistenceAdapter.java:908) > at > org.apache.activemq.store.amq.AMQPersistenceAdapter.start(AMQPersistenceAdapter.java:167) > at > org.apache.activemq.broker.BrokerService.createRegionBroker(BrokerService.java:1577) > at > org.apache.activemq.broker.BrokerService.createBroker(BrokerService.java:1530) > at > org.apache.activemq.broker.BrokerService.getBroker(BrokerService.java:540) > at > org.apache.activemq.broker.BrokerService.start(BrokerService.java:444) > at > org.apache.activemq.transport.vm.VMTransportFactory.doCompositeConnect(VMTransportFactory.java:119) > at > org.apache.activemq.transport.vm.VMTransportFactory.doConnect(VMTransportFactory.java:53) > at > org.apache.activemq.transport.TransportFactory.doConnect(TransportFactory.java:47) > at > org.apache.activemq.transport.TransportFactory.connect(TransportFactory.java:76) > at > org.apache.activemq.ActiveMQConnectionFactory.createTransport(ActiveMQConnectionFactory.java:235) > at > org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:250) > at > org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:222) > at > org.apache.activemq.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:170) > at > org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:188) > at > org.springframework.jms.listener.AbstractJmsListeningContainer.createSharedConnection(AbstractJmsListeningContainer.java:209) > at > org.springframework.jms.listener.AbstractJmsListeningContainer.establishSharedConnection(AbstractJmsListeningContainer.java:181) > at > org.springframework.jms.listener.AbstractJmsListeningContainer.doStart(AbstractJmsListeningContainer.java:345) > at > org.springframework.jms.listener.AbstractJmsListeningContainer.initialize(AbstractJmsListeningContainer.java:158) > at > org.springframework.jms.listener.AbstractJmsListeningContainer.afterPropertiesSet(AbstractJmsListeningContainer.java:133) > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1390) > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1359) > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:540) > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:485) > at java.security.AccessController.doPrivileged(Native Method) > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:455) > at > org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:251) > at > org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:169) > at > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:248) > at > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:170) > at > org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:413) > at > org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:735) > at > org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:369) > at > org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:96) > at > org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:44) > at > org.springframework.test.context.TestContext.buildApplicationContext(TestContext.java:198) > at > org.springframework.test.context.TestContext.getApplicationContext(TestContext.java:233) > at > org.springframework.test.context.support.DependencyInjectionTestExecutionListener.injectDependencies(DependencyInjectionTestExecutionListener.java:126) > at > org.springframework.test.context.support.DependencyInjectionTestExecutionListener.prepareTestInstance(DependencyInjectionTestExecutionListener.java:85) > at > org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:231) > at > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.createTest(SpringJUnit4ClassRunner.java:95) > at > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.invokeTestMethod(SpringJUnit4ClassRunner.java:139) > at > org.junit.internal.runners.JUnit4ClassRunner.runMethods(JUnit4ClassRunner.java:51) > at > org.junit.internal.runners.JUnit4ClassRunner$1.run(JUnit4ClassRunner.java:44) > at > org.junit.internal.runners.ClassRoadie.runUnprotected(ClassRoadie.java:27) > at > org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:37) > at > org.junit.internal.runners.JUnit4ClassRunner.run(JUnit4ClassRunner.java:42) > at > org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59) > at > org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:120) > at > org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:103) > at org.apache.maven.surefire.Surefire.run(Surefire.java:169) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350) > at > org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021) > > -----Original Message----- > From: Shelley, Ryan [mailto:[email protected]] > Sent: Tuesday, January 18, 2011 3:11 PM > To: '[email protected]' > Subject: ActiveMQ and NFS mounts > > I have a Continuous Integration system that runs on an NFS mount. I'm > getting errors about ActiveMQ not being able to obtain a lock, which I've > seen before in my code coverage tools, due to our NFS mount. > > The only broker we have running during a CI test case is a VM broker, so I > tried appending "?dataDirectory=/tmp" to the VM URI to move the journal to an > actual disk but I get the same failure (we aren't using a XML config for a VM > broker, so I'm trying to configure it through the URI). Is there something > I'm missing or something I can try to avoid the locks? I don't care that it > can't get a lock because it's just a CI test case, but it's causing our > entire build to fail. Any ideas? > > Thanks! > > -Ryan > > -- Stan Lewis FuseSource Email: [email protected] Web: http://fusesource.com Twitter: gashcrumb
