I changed my broker in my activemq.xml in servicemix like this:
<!-- ActiveMQ JMS Broker configuration -->
<amq:broker id="broker" depends-on="jmxServer" useShutdownHook="false"
persistent="false">
and then started it up. It went to 1444 and then stopped processing
anything. So my guess is that that doesn't help either. I think some of my
other services are able to keep sender so it seems like the producer flow
control is kicking in rather than my messages being expired like I want.
On Fri, Aug 22, 2008 at 7:43 AM, Gert Vanthienen
<[EMAIL PROTECTED]>wrote:
> Ryan,
>
> Odd... Just to be sure we are not looking at something completely different
> here, could you try to disable persistence on the broker alltogether to
> ensure that this fixes the problem? From the stacktraces, it looks like it
> is the SEDA flow itself that is hanging here (waiting for thread to become
> available after a sendSync to somewhere perhaps?)
>
> Regards,
>
> Gert
>
>
> Ryan Moquin wrote:
>
>> Just a little bit more. So everything I find on this issue, seems to
>> indicate I guess that since the JMS delivery mode is persistent by
>> default,
>> you should turn off persistence if you don't want a producer to block. So
>> since I don't want my service to block, nor do I want the messages
>> persistent, I configured my jms provider like this:
>>
>> <jms:provider service="not:notification-jms-service"
>> endpoint="notification"
>> pubSubDomain="true"
>> destinationName="topic.notication"
>> connectionFactory="#connectionFactory"
>> deliveryMode="1"
>> timeToLive="1000"/>
>>
>> As far as I can tell (since it's not documented what the values of
>> deliveryMode are), this should keep my producer from blocking, but it
>> still
>> does after it hits a little over a 1000 messages. Is there a way to turn
>> off persistence in my producers that work? I have not found a way to make
>> this happen.
>>
>> On Tue, Aug 19, 2008 at 7:54 PM, Ryan Moquin <[EMAIL PROTECTED]>
>> wrote:
>>
>>
>>
>>> Maybe this is the better way to ask this as I try to rationalize this
>>> behavior in my head:
>>>
>>> If you have a service unit that sends messages to a topic created by a
>>> servicemix-jms provider endpoint that has no consumers, shouldn't every
>>> message sent to it, be discarded? I'm not really sure I actually know
>>> what
>>> the problem is, since all my configuration attempts to fix it have
>>> failed.
>>> All I want at this point is any way possible to prevent the system from
>>> deadlocking. I'd be happy going back to exceptions being thrown all the
>>> time instead. At one point with Servicemix 3.2.2, we've processed lots
>>> and
>>> lots of messages (eventually Servicemix ran out of memory and crashed, no
>>> clue as to why that happened consistently every 2 hours, but that's
>>> another
>>> issue, I'm guessing because ActiveMQ never discards any messages). We
>>> haven't changed any of the servicemix configurations since then, so I'm
>>> not
>>> sure exactly why we can only process a small amount before Servicemix
>>> will
>>> hang from ActiveMQ blocking.
>>>
>>> I'm very puzzled at this point.
>>>
>>>
>>> On Tue, Aug 19, 2008 at 10:33 AM, Ryan Moquin <[EMAIL PROTECTED]
>>> >wrote:
>>>
>>>
>>>
>>>> Hello,
>>>>
>>>> So, due to my service units hanging under certain conditions when I try
>>>> to
>>>> send a JMS message using an injected DeliveryChannel, I switched to
>>>> using a
>>>> ServicemixClient I look up out of the servicemix JNDI context. This is
>>>> done
>>>> by several of my deployed services that are doing various pieces of
>>>> work.
>>>> I'm now noticing that while switching to a separate ServicemixClient has
>>>> fixed some of my issues, I'm now seeing certain services that send
>>>> messages
>>>> to a JMS service unit to send out of the syste, are also now deadlocking
>>>> around the same point and therefore my service halts. I did a thread
>>>> dump
>>>> to verify this and I see A LOT of threads that are waiting on a specific
>>>> lock with id: 0x1abeba80. You can see one of these below:
>>>>
>>>> "pool-flow.seda.servicemix-jms-thread-1157" prio=6 tid=0x3cee2400
>>>> nid=0x1a00 waiting on condition [0
>>>> x3869f000..0x3869fa98]
>>>> java.lang.Thread.State: WAITING (parking)
>>>> at sun.misc.Unsafe.park(Native Method)
>>>> - parking to wait for <0x1abeba80> (a
>>>> java.util.concurrent.locks.AbstractQueuedSynchronizer
>>>> $ConditionObject)
>>>> at
>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>>>> at
>>>>
>>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueue
>>>> dSynchronizer.java:1925)
>>>> at
>>>> java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:252)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImp
>>>> l.java:663)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:172)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:167)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
>>>> at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
>>>> at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>>>> at java.lang.Thread.run(Thread.java:619)
>>>>
>>>> My service itself is also locking on the same id, while trying to send a
>>>> message to the JMS consumer (I think is the right term):
>>>>
>>>> "QuartzScheduler_Worker-7" prio=6 tid=0x3a2f1400 nid=0x1050 waiting on
>>>> condition [0x37e2e000..0x37e2
>>>> fc18]
>>>> java.lang.Thread.State: WAITING (parking)
>>>> at sun.misc.Unsafe.park(Native Method)
>>>> - parking to wait for <0x1abeba80> (a
>>>> java.util.concurrent.locks.AbstractQueuedSynchronizer
>>>> $ConditionObject)
>>>> at
>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>>>> at
>>>>
>>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueue
>>>> dSynchronizer.java:1925)
>>>> at
>>>> java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:252)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImp
>>>> l.java:663)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:172)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:167)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
>>>> at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy.rejectedExecution(ThreadPoolExec
>>>> utor.java:1737)
>>>> at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
>>>> at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
>>>> at
>>>>
>>>> org.apache.servicemix.executors.impl.ExecutorImpl.execute(ExecutorImpl.java:43)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue.enqueue(SedaQueue.java:128)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.enqueuePacket(SedaFlow.java:182)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doSend(SedaFlow.java:162)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.AbstractFlow.send(AbstractFlow.java:123)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.DefaultBroker.sendExchangePacket(DefaultBroker.java:283)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.security.SecuredBroker.sendExchangePacket(SecuredBroker.java:88
>>>> )
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.container.JBIContainer.sendExchange(JBIContainer.java:830)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.doSend(DeliveryChannelImpl.java:3
>>>> 95)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.send(DeliveryChannelImpl.java:431
>>>> )
>>>> at
>>>>
>>>> org.apache.servicemix.jms.multiplexing.MultiplexingProviderProcessor.process(Multiplexing
>>>> ProviderProcessor.java:143)
>>>> at
>>>>
>>>> org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeCycle.java:538)
>>>> at
>>>>
>>>> org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBaseLifeCycle.java:4
>>>> 90)
>>>> at
>>>>
>>>> org.apache.servicemix.common.BaseLifeCycle.onMessageExchange(BaseLifeCycle.java:46)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImp
>>>> l.java:610)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:172)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:167)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
>>>> at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy.rejectedExecution(ThreadPoolExec
>>>> utor.java:1737)
>>>> at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
>>>> at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
>>>> at
>>>>
>>>> org.apache.servicemix.executors.impl.ExecutorImpl.execute(ExecutorImpl.java:43)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue.enqueue(SedaQueue.java:128)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.enqueuePacket(SedaFlow.java:182)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doSend(SedaFlow.java:162)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.flow.AbstractFlow.send(AbstractFlow.java:123)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.nmr.DefaultBroker.sendExchangePacket(DefaultBroker.java:283)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.security.SecuredBroker.sendExchangePacket(SecuredBroker.java:88
>>>> )
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.container.JBIContainer.sendExchange(JBIContainer.java:830)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.doSend(DeliveryChannelImpl.java:3
>>>> 95)
>>>> at
>>>>
>>>> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.send(DeliveryChannelImpl.java:431
>>>> )
>>>> at
>>>>
>>>> org.apache.servicemix.client.DefaultServiceMixClient.send(DefaultServiceMixClient.java:14
>>>> 7)
>>>> at
>>>>
>>>> org.apache.servicemix.client.DefaultServiceMixClient.send(DefaultServiceMixClient.java:21
>>>> 1)
>>>>
>>>> So my question is, is activemq freezing up for some reason? All my
>>>> problems I have, seem to somehow stem to activemq in various ways,
>>>> usually
>>>> all my freeze ups seem to be attributed to ActiveMQ freezing up for one
>>>> reason or another. If the above case is that ActiveMQ is freezing up
>>>> (it
>>>> happens over a period of time, I thought I read that ActiveMQ will block
>>>> when a topic or Queue fills up, which makes no sense to me in regards to
>>>> a
>>>> topic, since I thought that a topic doesn't hold messages unless they
>>>> are
>>>> marked durable), are there settings I can use to keep this from
>>>> happening?
>>>> There is no way I can ever trust this sort of system in production
>>>> unless I
>>>> can be sure that Servicemix/ActiveMQ isn't going to lock up on me after
>>>> a
>>>> couple hours.
>>>>
>>>> In case you want to see how I'm using the ServicemixClient, I'm not
>>>> doing
>>>> anything overly special, but here it is:
>>>>
>>>> if (client == null) {
>>>> ClientFactory factory = (ClientFactory) new
>>>> InitialContext().lookup(ClientFactory.DEFAULT_JNDI_NAME);
>>>> client = factory.createClient();
>>>> }
>>>>
>>>> if (destinationQname == null) {
>>>> destinationQname = new QName(getDestinationNamespace(),
>>>> getDestinationService());
>>>> }
>>>> EndpointResolver resolver =
>>>> client.createResolverForService(destinationQname);
>>>> client.send(resolver, null, null, new
>>>> StringSource(writer.toString()));
>>>>
>>>> I'm not missing anything in the above code am I?
>>>> Thanks,
>>>> Ryan
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>>
>
>
