Maybe this is the better way to ask this as I try to rationalize this
behavior in my head:
If you have a service unit that sends messages to a topic created by a
servicemix-jms provider endpoint that has no consumers, shouldn't every
message sent to it, be discarded? I'm not really sure I actually know what
the problem is, since all my configuration attempts to fix it have failed.
All I want at this point is any way possible to prevent the system from
deadlocking. I'd be happy going back to exceptions being thrown all the
time instead. At one point with Servicemix 3.2.2, we've processed lots and
lots of messages (eventually Servicemix ran out of memory and crashed, no
clue as to why that happened consistently every 2 hours, but that's another
issue, I'm guessing because ActiveMQ never discards any messages). We
haven't changed any of the servicemix configurations since then, so I'm not
sure exactly why we can only process a small amount before Servicemix will
hang from ActiveMQ blocking.
I'm very puzzled at this point.
On Tue, Aug 19, 2008 at 10:33 AM, Ryan Moquin <[EMAIL PROTECTED]>wrote:
> Hello,
>
> So, due to my service units hanging under certain conditions when I try to
> send a JMS message using an injected DeliveryChannel, I switched to using a
> ServicemixClient I look up out of the servicemix JNDI context. This is done
> by several of my deployed services that are doing various pieces of work.
> I'm now noticing that while switching to a separate ServicemixClient has
> fixed some of my issues, I'm now seeing certain services that send messages
> to a JMS service unit to send out of the syste, are also now deadlocking
> around the same point and therefore my service halts. I did a thread dump
> to verify this and I see A LOT of threads that are waiting on a specific
> lock with id: 0x1abeba80. You can see one of these below:
>
> "pool-flow.seda.servicemix-jms-thread-1157" prio=6 tid=0x3cee2400
> nid=0x1a00 waiting on condition [0
> x3869f000..0x3869fa98]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x1abeba80> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueue
> dSynchronizer.java:1925)
> at
> java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:252)
> at
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImp
> l.java:663)
> at
> org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:172)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:167)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:619)
>
> My service itself is also locking on the same id, while trying to send a
> message to the JMS consumer (I think is the right term):
>
> "QuartzScheduler_Worker-7" prio=6 tid=0x3a2f1400 nid=0x1050 waiting on
> condition [0x37e2e000..0x37e2
> fc18]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x1abeba80> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueue
> dSynchronizer.java:1925)
> at
> java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:252)
> at
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImp
> l.java:663)
> at
> org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:172)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:167)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
> at
> java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy.rejectedExecution(ThreadPoolExec
> utor.java:1737)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> at
> org.apache.servicemix.executors.impl.ExecutorImpl.execute(ExecutorImpl.java:43)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue.enqueue(SedaQueue.java:128)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.enqueuePacket(SedaFlow.java:182)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doSend(SedaFlow.java:162)
> at
> org.apache.servicemix.jbi.nmr.flow.AbstractFlow.send(AbstractFlow.java:123)
> at
> org.apache.servicemix.jbi.nmr.DefaultBroker.sendExchangePacket(DefaultBroker.java:283)
> at
> org.apache.servicemix.jbi.security.SecuredBroker.sendExchangePacket(SecuredBroker.java:88
> )
> at
> org.apache.servicemix.jbi.container.JBIContainer.sendExchange(JBIContainer.java:830)
> at
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.doSend(DeliveryChannelImpl.java:3
> 95)
> at
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.send(DeliveryChannelImpl.java:431
> )
> at
> org.apache.servicemix.jms.multiplexing.MultiplexingProviderProcessor.process(Multiplexing
> ProviderProcessor.java:143)
> at
> org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeCycle.java:538)
> at
> org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBaseLifeCycle.java:4
> 90)
> at
> org.apache.servicemix.common.BaseLifeCycle.onMessageExchange(BaseLifeCycle.java:46)
> at
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImp
> l.java:610)
> at
> org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:172)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:167)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
> at
> java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy.rejectedExecution(ThreadPoolExec
> utor.java:1737)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> at
> org.apache.servicemix.executors.impl.ExecutorImpl.execute(ExecutorImpl.java:43)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue.enqueue(SedaQueue.java:128)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.enqueuePacket(SedaFlow.java:182)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doSend(SedaFlow.java:162)
> at
> org.apache.servicemix.jbi.nmr.flow.AbstractFlow.send(AbstractFlow.java:123)
> at
> org.apache.servicemix.jbi.nmr.DefaultBroker.sendExchangePacket(DefaultBroker.java:283)
> at
> org.apache.servicemix.jbi.security.SecuredBroker.sendExchangePacket(SecuredBroker.java:88
> )
> at
> org.apache.servicemix.jbi.container.JBIContainer.sendExchange(JBIContainer.java:830)
> at
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.doSend(DeliveryChannelImpl.java:3
> 95)
> at
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.send(DeliveryChannelImpl.java:431
> )
> at
> org.apache.servicemix.client.DefaultServiceMixClient.send(DefaultServiceMixClient.java:14
> 7)
> at
> org.apache.servicemix.client.DefaultServiceMixClient.send(DefaultServiceMixClient.java:21
> 1)
>
> So my question is, is activemq freezing up for some reason? All my
> problems I have, seem to somehow stem to activemq in various ways, usually
> all my freeze ups seem to be attributed to ActiveMQ freezing up for one
> reason or another. If the above case is that ActiveMQ is freezing up (it
> happens over a period of time, I thought I read that ActiveMQ will block
> when a topic or Queue fills up, which makes no sense to me in regards to a
> topic, since I thought that a topic doesn't hold messages unless they are
> marked durable), are there settings I can use to keep this from happening?
> There is no way I can ever trust this sort of system in production unless I
> can be sure that Servicemix/ActiveMQ isn't going to lock up on me after a
> couple hours.
>
> In case you want to see how I'm using the ServicemixClient, I'm not doing
> anything overly special, but here it is:
>
> if (client == null) {
> ClientFactory factory = (ClientFactory) new
> InitialContext().lookup(ClientFactory.DEFAULT_JNDI_NAME);
> client = factory.createClient();
> }
>
> if (destinationQname == null) {
> destinationQname = new QName(getDestinationNamespace(),
> getDestinationService());
> }
> EndpointResolver resolver =
> client.createResolverForService(destinationQname);
> client.send(resolver, null, null, new
> StringSource(writer.toString()));
>
> I'm not missing anything in the above code am I?
> Thanks,
> Ryan
>
