Hey Maarten, I am observing the same behavior in my AMQ application. Was your problem resolved? I've tried a few different connection factory approach to no avail :(
Thanks, Elliot Maarten_D wrote: > > Hi Rob, > Sorry, I'd killed that particular JVM before I read your request, and the > error hasn't reoccurred since then. > Something else that I've been thinking about since reading a > http://tmielke.blogspot.com/2009/12/using-spring-jms-template-for-sending.html > blog post : we used the ActiveMQ PooledConnectionFactory for establishing > all connections to the broker, for the Spring message listener containers > as well as for the JMS templates. > After diving into the code of the PooledconnectionFactory, this seems to > have been a bad idea. When a connection is requested, the PCF simply > creates a new one and hands it out. When the pool is full, it returns the > first connection in its list (ie the first one it created), removes it > from the top of the list and adds it to the bottom. This means that, if > your listeners also get connections from this pool (and remember, > listeners hold on to their connections), a whole bunch of JMS template > calls will be made where the template is sending messages using the same > connector used by a listener. > I'm not too sure about the details, but when you introduce > producerflowcontrol into this picture, I can imagine how a kind of > deadlock can occur where eventually all producers on all connections are > throttled, leaving no one able to send any messages. > Does this sound like a plausible scenario? > > We've modified our config and given all listener containers a connection > that's not in the pool, and are now running another test. I'll post the > results. > > Regards, > Maarten > > PS. Of course, you might say we should've know about this beforehand, as > the PCF javadoc says that its not really meant for consumers. The > alternative it offers is Jencks, although that project has been dead for a > while, to the point where even the homepage is now a spam site > (jencks.org). So clearly that isn't a viable alternative. > > > rajdavies wrote: >> >> Can you take a thread dump whilst its in this state - and send us the >> output ? >> >> thanks, >> >> Rob >> On 21 Jan 2010, at 17:26, Maarten_D wrote: >> >>> >>> O, and I forgot to mention I also turned on async sends >>> (jms.useAsyncSend=true) >>> >>> Maarten_D wrote: >>>> >>>> I've now changed my activemq.xml to the listing below, made the >>>> session >>>> transacted and set the acknowledge mode to SESSION_TRANSACTED. >>>> >>>> Things were going well for me for a while, with the system >>>> processing 3,2 >>>> million messages without a hitch, and then everything stopped >>>> because the >>>> first component in the chain got lots of these: >>>> >>>> javax.jms.InvalidClientIDException: Broker: broker - Client: >>>> ID:rhost-59116-1263927611185-1:445 already connected from / >>>> 127.0.0.1:56560 >>>> >>>> And for an hour now, since it stopped processing messages, the >>>> broker has >>>> been eating up almost 100% of the cpu for some reason I can't quite >>>> fathom >>>> (disk utilization is very low, and there is no message traffic >>>> passing >>>> through the broker). >>>> >>>> <beans xmlns="http://www.springframework.org/schema/beans"; >>>> xmlns:amq="http://activemq.apache.org/schema/core"; >>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >>>> xsi:schemaLocation="http://www.springframework.org/schema/beans >>>> http://www.springframework.org/schema/beans/spring-beans-2.0.xsd >>>> http://activemq.apache.org/schema/core >>>> http://activemq.apache.org/schema/core/activemq-core.xsd >>>> http://mortbay.com/schemas/jetty/1.0 http://jetty.mortbay.org/jetty.xsd >>>> "> >>>> >>>> <bean >>>> class >>>> = >>>> "org >>>> .springframework.beans.factory.config.PropertyPlaceholderConfigurer"> >>>> <property name="location" value="file:/etc/broker.properties" /> >>>> </bean> >>>> >>>> <broker id="broker" useJmx="true" brokerName="${broker.name}" >>>> start="true" xmlns="http://activemq.apache.org/schema/core"; >>>> dataDirectory="${activemq.data}"> >>>> >>>> <destinationPolicy> >>>> <policyMap> >>>> <policyEntries> >>>> <policyEntry queue=">" memoryLimit="32mb" >>>> strictOrderDispatch="true" producerFlowControl="false"> >>>> <pendingQueuePolicy> >>>> <vmQueueCursor /> >>>> </pendingQueuePolicy> >>>> </policyEntry> >>>> <policyEntry topic=">" memoryLimit="32mb" >>>> producerFlowControl="true" /> >>>> </policyEntries> >>>> </policyMap> >>>> </destinationPolicy> >>>> >>>> <managementContext> >>>> <managementContext useMBeanServer="true" >>>> jmxDomainName="org.apache.activemq" >>>> createMBeanServer="true" >>>> createConnector="false" >>>> connectorPort="1100" >>>> connectorPath="/jmxrmi"/> >>>> </managementContext> >>>> >>>> <persistenceAdapter> >>>> <kahaDB directory="${activemq.data}/${broker.name}" >>>> journalMaxFileLength="32mb" >>>> enableJournalDiskSyncs="false" >>>> indexWriteBatchSize="1000" >>>> indexCacheSize="1000"/> >>>> </persistenceAdapter> >>>> >>>> <systemUsage> >>>> <systemUsage> >>>> <memoryUsage> >>>> <memoryUsage limit="512mb" /> >>>> </memoryUsage> >>>> </systemUsage> >>>> </systemUsage> >>>> >>>> <transportConnectors> >>>> <transportConnector uri="nio://0.0.0.0:61616" /> >>>> </transportConnectors> >>>> </broker> >>>> >>>> <jetty xmlns="http://mortbay.com/schemas/jetty/1.0";> >>>> <connectors> >>>> <nioConnector port="61617"/> >>>> </connectors> >>>> <handlers> >>>> <webAppContext contextPath="/admin" >>>> resourceBase="${activemq.base}/webapps/admin" logUrlOnStart="true"/> >>>> </handlers> >>>> </jetty> >>>> </beans> >>>> >>> >>> -- >>> View this message in context: >>> http://old.nabble.com/50k-%2B-messages-stuck-in-queue-with-all-consumers-blocking-on-receive-tp27162095p27261393.html >>> Sent from the ActiveMQ - User mailing list archive at Nabble.com. >>> >> >> Rob Davies >> http://twitter.com/rajdavies >> I work here: http://fusesource.com >> My Blog: http://rajdavies.blogspot.com/ >> I'm writing this: http://www.manning.com/snyder/ >> >> >> >> >> >> >> > > -- View this message in context: http://old.nabble.com/50k-%2B-messages-stuck-in-queue-with-all-consumers-blocking-on-receive-tp27162095p27664172.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
