It's still failing sometimes on my machine: :-( Results : Failed tests: JpaProducerConcurrentTest.testConcurrentProducers:45->doSendMessages:66->CamelTestSupport.assertMockEndpointsSatisfied:682 mock://result Received message count. Expected: <10> but was: <8>
Tests run: 49, Failures: 1, Errors: 0, Skipped: 0 [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ Best, Christian ----------------- Software Integration Specialist Apache Member V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer Apache Incubator PMC Member https://www.linkedin.com/pub/christian-mueller/11/551/642 On Sat, Oct 12, 2013 at 11:07 PM, Babak Vahdat <[email protected]>wrote: > Yeah indeed a deadlock inside the OpenJPA code is the root cause of the > problem (org.apache.openjpa.kernel.BrokerImpl#lock()). I just commited a > fix > regarding this, so let's keep an eye on this test by the next builds. > Following a screenshot of the proven deadlock through jconsole: > > < > http://camel.465427.n5.nabble.com/file/n5741448/Screen_Shot_2013-10-12_at_11.02.58_PM.png > > > > You can enforce the deadlock if you would make use of a high log level > (log4j.properties), e.g. TRACE instead of just INFO. > > Babak > > > Claus Ibsen-2 wrote > > It could be due an OpenJPA upgrade. These tests used to work better in > the > > past. > > > > On Sat, Oct 12, 2013 at 10:05 AM, Babak Vahdat > > < > > > babak.vahdat@ > > > > wrote: > >> The test failed again on the CI-Server. I will look into this in the > next > >> days to see if I can find the root cause as on my box I can't reproduce > >> this. > >> > >> Babak > >> > >> > >> Babak Vahdat wrote > >>> Hi > >>> > >>> @Christian I believe I've fixed the JPA test now, could you please > check > >>> to see if you can still reproduce the same exception you mentioned > >>> below? > >>> > >>> Babak > >>> Christian Mueller wrote > >>>> Today, it looks much better [1]. We are trying to fix the unit tests > >>>> which > >>>> are failing from time to time on slower boxes. But this isn't so easy > >>>> for > >>>> all of them... > >>>> > >>>> I had a look at > >>>> > org.apache.camel.processor.jpa.JpaProducerConcurrentTest.testConcurrentProducers. > >>>> A few times I get the following exception which results in missing > >>>> exchanges in the mock:result endpoint: > >>>> > >>>> 2013-10-11 14:01:10,623 [pool-2-thread-2] ERROR > >>>> DefaultErrorHandler - Failed delivery for (MessageId: > >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4 on > >>>> ExchangeId: > >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9). > Exhausted > >>>> after delivery attempt: 1 caught: > >>>> org.springframework.dao.InvalidDataAccessApiUsageException: This > >>>> operation > >>>> cannot be performed while a Transaction is active.; nested exception > is > >>>> > > <openjpa-2.2.2-r422266:1468616 nonfatal user error> > >>>> org.apache.openjpa.persistence.InvalidStateException: This operation > >>>> cannot > >>>> be performed while a Transaction is active. > >>>> > >>>> Message History > >>>> > --------------------------------------------------------------------------------------------------------------------------------------- > >>>> RouteId ProcessorId > >>>> Processor > >>>> Elapsed (ms) > >>>> [route1 ] [route1 ] > >>>> [direct://start > >>>> ] [ 68] > >>>> [route1 ] [to1 ] > >>>> [jpa://org.apache.camel.examples.SendEmail > >>>> ] [ 67] > >>>> > >>>> Exchange > >>>> > --------------------------------------------------------------------------------------------------------------------------------------- > >>>> Exchange[ > >>>> Id > >>>> ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-9 > >>>> ExchangePattern InOnly > >>>> Headers > >>>> > {breadcrumbId=ID-christians-macbook-pro-fritz-box-54975-1381492869188-0-4, > >>>> > CamelEntityManager=org.apache.openjpa.persistence.EntityManagerImpl@6e453dd5 > , > >>>> CamelRedelivered=false, CamelRedeliveryCounter=0} > >>>> BodyType org.apache.camel.examples.SendEmail > >>>> Body SendEmail[id: 102 address: null] > >>>> ] > >>>> > >>>> Stacktrace > >>>> > --------------------------------------------------------------------------------------------------------------------------------------- > >>>> org.springframework.dao.InvalidDataAccessApiUsageException: This > >>>> operation > >>>> cannot be performed while a Transaction is active.; nested exception > is > >>>> > > <openjpa-2.2.2-r422266:1468616 nonfatal user error> > >>>> org.apache.openjpa.persistence.InvalidStateException: This operation > >>>> cannot > >>>> be performed while a Transaction is active. > >>>> at > >>>> > org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:293) > >>>> at > >>>> > org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.convertException(ExtendedEntityManagerCreator.java:501) > >>>> at > >>>> > org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerSynchronization.afterCommit(ExtendedEntityManagerCreator.java:482) > >>>> at > >>>> > org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCommit(TransactionSynchronizationUtils.java:133) > >>>> at > >>>> > org.springframework.transaction.support.TransactionSynchronizationUtils.triggerAfterCommit(TransactionSynchronizationUtils.java:121) > >>>> at > >>>> > org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCommit(AbstractPlatformTransactionManager.java:951) > >>>> at > >>>> > org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:797) > >>>> at > >>>> > org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724) > >>>> at > >>>> > org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:148) > >>>> at > >>>> > org.apache.camel.component.jpa.JpaProducer.process(JpaProducer.java:53) > >>>> at > >>>> > org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61) > >>>> at > >>>> > org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:132) > >>>> at > >>>> > org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:307) > >>>> at > >>>> > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:127) > >>>> at > >>>> > org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398) > >>>> at > >>>> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191) > >>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:118) > >>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:80) > >>>> at > >>>> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191) > >>>> at > >>>> > org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:56) > >>>> at > >>>> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191) > >>>> at > >>>> > org.apache.camel.processor.UnitOfWorkProducer.process(UnitOfWorkProducer.java:73) > >>>> at > >>>> > org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:378) > >>>> at > >>>> > org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:1) > >>>> at > >>>> > org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:242) > >>>> at > >>>> > org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:346) > >>>> at > org.apache.camel.impl.ProducerCache.send(ProducerCache.java:184) > >>>> at > >>>> > org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:124) > >>>> at > >>>> > org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:137) > >>>> at > >>>> > org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:144) > >>>> at > >>>> > org.apache.camel.processor.jpa.JpaProducerConcurrentTest$1.call(JpaProducerConcurrentTest.java:58) > >>>> at > >>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138) > >>>> at > >>>> > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > >>>> at > >>>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > >>>> at java.lang.Thread.run(Thread.java:680) > >>>> Caused by: > >>>> > > <openjpa-2.2.2-r422266:1468616 nonfatal user error> > >>>> org.apache.openjpa.persistence.InvalidStateException: This operation > >>>> cannot > >>>> be performed while a Transaction is active. > >>>> at > org.apache.openjpa.kernel.BrokerImpl.begin(BrokerImpl.java:1369) > >>>> at > >>>> > org.apache.openjpa.kernel.DelegatingBroker.begin(DelegatingBroker.java:925) > >>>> at > >>>> > org.apache.openjpa.persistence.EntityManagerImpl.begin(EntityManagerImpl.java:565) > >>>> at > >>>> > org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.enlistInCurrentTransaction(ExtendedEntityManagerCreator.java:422) > >>>> at > >>>> > org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.doJoinTransaction(ExtendedEntityManagerCreator.java:399) > >>>> at > >>>> > org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:351) > >>>> at com.sun.proxy.$Proxy18.joinTransaction(Unknown Source) > >>>> at > >>>> > org.apache.camel.component.jpa.JpaProducer$1.doInTransaction(JpaProducer.java:55) > >>>> at > >>>> > org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:131) > >>>> ... 27 more > >>>> > >>>> which is throwing from the line 55 in > >>>> org.apache.camel.component.jpa.JpaProducer > >>>> entityManager.joinTransaction(); > >>>> > >>>> I think this is a concurrency issue we have to investigate (not sure > >>>> whether the issue is in Camel, Spring, OpenJPA, ...). Will open a JIRA > >>>> for > >>>> it. > >>>> > >>>> [1] https://builds.apache.org/job/Camel.trunk.fulltest/1562/ > >>>> > >>>> Best, > >>>> > >>>> Christian > >>>> ----------------- > >>>> > >>>> Software Integration Specialist > >>>> > >>>> Apache Member > >>>> V.P. Apache Camel | Apache Camel PMC Member | Apache Camel committer > >>>> Apache Incubator PMC Member > >>>> > >>>> https://www.linkedin.com/pub/christian-mueller/11/551/642 > >>>> > >>>> > >>>> On Thu, Oct 10, 2013 at 12:28 PM, James Carman > >>>> < > >> > >>>> james@ > >> > >>>> >wrote: > >>>> > >>>>> There are quite a few broken tests. Jetty seems to be hosed. JPA > has > >>>>> a > >>>>> failed test. CDI has issues. It looks like JMS and FTP even decided > >>>>> to > >>>>> join the party. This has been going on for a while now. This > >>>>> particular > >>>>> job hasn't been "green" since 9/29. Should we stop working other > >>>>> issues > >>>>> and get ourselves whole again? > >>>>> > >>>>> https://builds.apache.org/job/Camel.trunk.fulltest/1561/ > >>>>> > >> > >> > >> > >> > >> > >> -- > >> View this message in context: > >> http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741403.html > >> Sent from the Camel Development mailing list archive at Nabble.com. > > > > > > > > -- > > Claus Ibsen > > ----------------- > > Red Hat, Inc. > > Email: > > > cibsen@ > > > Twitter: davsclaus > > Blog: http://davsclaus.com > > Author of Camel in Action: http://www.manning.com/ibsen > > > > > > -- > View this message in context: > http://camel.465427.n5.nabble.com/Build-failures-tp5741273p5741448.html > Sent from the Camel Development mailing list archive at Nabble.com. >
