I found the issue just this time. It worked fine for about two month with the ActiveMQ M/S cluster. I've thought of the network glitch; What I feel strange most is there is no any error/exception on both the ActiveMQ client side and server side. I went through the related codes and found there is a FutureResponse.getResult method with parameter 'timeout'. Maybe this method is more suitable for getting the result. Not very sure, hehe.
At 2013-07-16 20:23:01,"Christian Posta" <christian.po...@gmail.com> wrote: >Is this something that happens all the time? Maybe your network had a >glitch and didn't deliver the response back to the client. > > >On Tue, Jul 16, 2013 at 8:18 AM, lzr <jsw...@163.com> wrote: > >> I just check the configuration file, actually I've disabled the flow >> control: >> >> >> <destinationPolicy> >> <policyMap> >> <policyEntries> >> <policyEntry topic=">" producerFlowControl="false" >> memoryLimit="100mb"> >> <pendingSubscriberPolicy> >> <vmCursor /> >> </pendingSubscriberPolicy> >> </policyEntry> >> <policyEntry queue=">" producerFlowControl="false" >> memoryLimit="100mb"> >> <!-- Use VM cursor for better latency >> For more information, see: >> >> >> http://activemq.apache.org/message-cursors.html >> >> >> <pendingQueuePolicy> >> <vmQueueCursor/> >> </pendingQueuePolicy> >> --> >> </policyEntry> >> </policyEntries> >> </policyMap> >> </destinationPolicy> >> >> >> >> >> >> >> >> >> >> >> At 2013-07-16 20:15:11,"Christian Posta" <christian.po...@gmail.com> >> wrote: >> >Yes, on the first time PFC kicked in you would see the log messages. Not >> on >> >subsequent times. >> > >> > >> >On Tue, Jul 16, 2013 at 8:01 AM, lzr <jsw...@163.com> wrote: >> > >> >> Based on my experiences if the issue caused by the producer flow control >> >> configuration, the logs will have some warning messages, right? In fact >> I >> >> didn't found any warning or errors in the activemq.log. >> >> Anyway, I'll check the configuration files right now. >> >> >> >> >> >> thanks lot, >> >> Zhuran. >> >> >> >> >> >> At 2013-07-16 19:36:44,"Christian Posta" <christian.po...@gmail.com> >> >> wrote: >> >> >> >> >This means the client didn't' receive a response from the broker. >> >> >Did the broker producer flow control this client? ie, was the >> destination >> >> >to which the client sending full? >> >> > >> >> > >> >> >On Tue, Jul 16, 2013 at 7:23 AM, lzr <jsw...@163.com> wrote: >> >> > >> >> >> Dear all, >> >> >> >> >> >> >> >> >> I found a one of the ActiveMQ thread got stuck unexpectedly for >> several >> >> >> days. Customer was very angry to the ActiveMQ for the bad behavior. >> Did >> >> >> anybody experienced this issue? It would be much appreciated if any >> >> clue!!! >> >> >> My application is normal Java JMS application like the following: >> >> >> new ActiveMQConnectionFactory. >> >> >> create Connection; >> >> >> conn.start. >> >> >> create session. >> >> >> create destination, producer and message >> >> >> send messages >> >> >> commit; >> >> >> conn.close. >> >> >> >> >> >> >> >> >> The follows is the stack when I looked into it with debug tools: >> >> >> >> >> >> sun.misc.Unsafe.park(Native Method) >> >> >> >> >> >> java.util.concurrent.locks.LockSupport.park(LockSupport.java:169) >> >> >> >> >> >> >> >> >> >> >> >> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1787) >> >> >> >> >> >> >> >> >> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:356) >> >> >> >> >> >> >> >> >> >> >> >> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40) >> >> >> >> >> >> >> >> >> >> >> >> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80) >> >> >> >> >> >> >> >> >> >> >> >> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233) >> >> >> >> >> >> >> >> >> >> >> >> org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339) >> >> >> >> >> >> >> >> >> org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:488) >> >> >> >> >> >> >> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:487) >> >> >> >> >> >> >> nc.bs.mqv2.mmgp.mq.UFMQProcessor.getConnection(UFMQProcessor.java:462) >> >> >> >> >> >> >> >> >> >> >> >> nc.bs.mqv2.mmgp.mq.UFMQProcessor.sendSerializedObjMsg(UFMQProcessor.java:128) >> >> >> >> >> >> >> >> >> >> >> >> nc.bs.mqv2.meam.mq.sendbasedatas.AbstractMeamBDSyncService.sendBaseData(AbstractMeamBDSyncService.java:64) >> >> >> >> >> >> >> >> >> >> >> >> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.dosendDataProcess(AbstractBDServerSendService.java:213) >> >> >> >> >> >> >> >> >> >> >> >> nc.bs.mqv2.mes.bd.mq.basedoc.AbstractBDServerSendService.sendDatas(AbstractBDServerSendService.java:62) >> >> >> >> >> >> >> >> >> >> >> >> nc.impl.mqv2.mes.bd.mq.SyncSendServiceImpl.ncserverSendDatas(SyncSendServiceImpl.java:58) >> >> >> >> >> >> nc.gejb.meam.cmt.MeamEjbBean.ncserverSendDatas(MeamEjbBean.java:8741) >> >> >> >> >> >> >> >> >> nc.gejb.meam.cmt.EJSLocalStatelessnc_gejb_meam_cmt_Meam_b44269e7.ncserverSendDatas(Unknown >> >> >> Source) >> >> >> >> >> >> sun.reflect.GeneratedMethodAccessor447.invoke(Unknown Source) >> >> >> >> >> >> >> >> >> >> >> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> >> >> >> >> java.lang.reflect.Method.invoke(Method.java:615) >> >> >> >> >> >> >> >> >> >> >> >> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56) >> >> >> >> >> >> >> >> >> >> >> >> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34) >> >> >> >> >> >> $Proxy58.ncserverSendDatas(Unknown Source) >> >> >> >> >> >> >> >> >> >> >> >> nc.bs.mqv2.meam.mq.task.DoSendBaseDataTaskUtil.runTask(DoSendBaseDataTaskUtil.java:63) >> >> >> >> >> >> >> >> >> >> >> >> nc.bs.mqv2.meam.mq.task.NCServerBaseDatasSendTask.executeTask(NCServerBaseDatasSendTask.java:26) >> >> >> >> >> >> >> >> >> >> >> >> nc.bs.pub.taskcenter.BgBizsPluginProcess.callBusinessPlugin(BgBizsPluginProcess.java:43) >> >> >> >> >> >> nc.bs.pub.pa.read.TaskBodyBase.callBizsPlugin(TaskBodyBase.java:188) >> >> >> >> >> >> >> nc.bs.pub.pa.read.TaskBodyBase.callBusinessPlugin(TaskBodyBase.java:174) >> >> >> >> >> >> nc.bs.pub.pa.read.TaskBodyBase.createMetaMsg(TaskBodyBase.java:95) >> >> >> >> >> >> >> >> >> nc.bs.pub.pa.read.TaskBodyBase.createAndSendMessage(TaskBodyBase.java:41) >> >> >> >> >> >> >> >> >> nc.bs.pub.taskcenter.BackgroundTaskBody.execute(BackgroundTaskBody.java:47) >> >> >> >> >> >> >> >> >> >> >> >> nc.impl.uap.scheduler.ExecutorServiceImpl.executeTask_RequiresNew(ExecutorServiceImpl.java:27) >> >> >> >> >> >> >> >> >> >> >> >> nc.itf.uap.mw.MwBizEJBEjbBean.executeTask_RequiresNew(MwBizEJBEjbBean.java:1682) >> >> >> >> >> >> >> >> >> nc.itf.uap.mw.EJSLocalStatelessnc_itf_uap_mw_MwBizEJB_f98e92a8.executeTask_RequiresNew(Unknown >> >> >> Source) >> >> >> >> >> >> sun.reflect.GeneratedMethodAccessor1036.invoke(Unknown Source) >> >> >> >> >> >> >> >> >> >> >> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> >> >> >> >> java.lang.reflect.Method.invoke(Method.java:615) >> >> >> >> >> >> >> >> >> >> >> >> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:56) >> >> >> >> >> >> >> >> >> >> >> >> nc.bs.framework.ejb.AbstractEJBServiceHandler.invoke(AbstractEJBServiceHandler.java:34) >> >> >> >> >> >> $Proxy39.executeTask_RequiresNew(Unknown Source) >> >> >> >> >> >> >> nc.bs.uap.scheduler.impl.TaskExecutor.executeTask(TaskExecutor.java:110) >> >> >> >> >> >> nc.bs.uap.scheduler.impl.TaskExecutor.run(TaskExecutor.java:75) >> >> >> >> >> >> nc.bs.framework.execute.Executor$1.run(Executor.java:80) >> >> >> >> >> >> >> >> >> nc.bs.framework.execute.commonj.impl.WorkAdaptor.run(WorkAdaptor.java:29) >> >> >> >> >> >> com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:1114) >> >> >> >> >> >> >> >> >> >> >> >> com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:195) >> >> >> >> >> >> com.ibm.ws.asynchbeans.CJWorkItemImpl.run(CJWorkItemImpl.java:187) >> >> >> >> >> >> com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1469) >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> Best regards, >> >> >> >> >> >> Zhuran Li >> >> > >> >> > >> >> > >> >> > >> >> >-- >> >> >*Christian Posta* >> >> >http://www.christianposta.com/blog >> >> >twitter: @christianposta >> >> >> > >> > >> > >> >-- >> >*Christian Posta* >> >http://www.christianposta.com/blog >> >twitter: @christianposta >> > > > >-- >*Christian Posta* >http://www.christianposta.com/blog >twitter: @christianposta
