Re: java.lang.OutOfMemoryError: unable to create new native thread

[xabhi]

>>Are you sure about that -Xss1024m? That would set the thread stack size to
a 
monstrous 1GB. Perhaps you wanted to configure the initial heap size (-Xms) 
there but made a typo?

Thanks for the help. The framework that launches the Java application was
incorrectly setting default stack size to 1024m.

Thanks,
Abhi



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: java.lang.OutOfMemoryError: unable to create new native thread

[xabhi]

It's at 64-bit system/JVM. 

The process limits from /proc/pid/limits are:
Max processes 1634116341   
processes 
Max open files6553665536files   

The JVM flags used are : -Xmx8g -Xss1024m 

The default value in JVM is:
amq...@stokes31.toa(nyc):~> java -XX:+PrintFlagsFinal -version | grep
ThreadStackSize
 intx ThreadStackSize   = 1024  
 
{pd product}
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

The OS limits and heap sizes seem more that enough to me. I have another
instance that runs with same configuration and hasn't exhibited this
behavior. What other factors should i look at for more information next time
this issue occurs. 

How can i increase the native memory space if thats what is causing this? 




--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

java.lang.OutOfMemoryError: unable to create new native thread

[xabhi]

Hi

I am using ActiveMQ v5.14.4 and have observed below multiple times now. I am
using NIO connector so thread count is not high (~203 threads when this
issue was happening). 

Exception in thread "ActiveMQ InactivityMonitor WriteCheckTimer"
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
at
org.apache.activemq.transport.AbstractInactivityMonitor.writeCheck(AbstractInactivityMonitor.java:189)
at
org.apache.activemq.transport.AbstractInactivityMonitor$3.run(AbstractInactivityMonitor.java:153)
at
org.apache.activemq.thread.SchedulerTimerTask.run(SchedulerTimerTask.java:33)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)

which then triggers following warnings and no new clients are able to
connect after this.

[20180702 06:49:50.829 EDT (ActiveMQ NIO Worker 8)
org.apache.activemq.broker.TransportConnection#serviceTransportException 243
WARN] - Transport Connection to: tcp://10.219.31.188:34470 failed:
java.io.IOException: Timer already cancelled. 
[20180702 06:49:50.945 EDT (ActiveMQ NIO Worker 7)
org.apache.activemq.broker.TransportConnection#serviceTransportException 243
WARN] - Transport Connection to: tcp://10.219.31.184:41305 failed:
java.io.IOException: Timer already cancelled. 
[20180702 06:49:50.945 EDT (ActiveMQ NIO Worker 5)
org.apache.activemq.broker.TransportConnection#serviceTransportException 243
WARN] - Transport Connection to: tcp://10.219.31.189:41305 failed:
java.io.IOException: Timer already cancelled. 
[20180702 06:49:50.945 EDT (ActiveMQ NIO Worker 3)
org.apache.activemq.broker.TransportConnection#serviceTransportException 243
WARN] - Transport Connection to: tcp://10.218.24.250:46430 failed:
java.io.IOException: Timer already cancelled.

What can cause this issue to happen? 203 threads doesn't seem too high. I
also found this related JIRA- https://issues.apache.org/jira/browse/AMQ-5543
   which seems to ahve fixed
this but i am still seeing this issue?

How should i go about fixing this? Does this look like a bug or a
configuration problem?

Thanks,
Abhi 



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: Frequently getting error in ActiveMQ and ActiveMQ gets stopped

[xabhi]

you can take a look at this wiki page -
http://activemq.apache.org/activemq-inactivitymonitor.html
and increase 'wireFormat.maxInactivityDuration' to a higher value.

Thanks,
Abhi



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: Artemis Kerberos support for openwire and stomp protocol

[xabhi]

Thanks for the reply Tim. 

Is there a plan to extend Kerberos Authentication to STOMP and other
protocols in Artemis? Should i log a JIRA ticket for this? 

Thanks,
Abhi



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Artemis Kerberos support for openwire and stomp protocol

[xabhi]

Hi,

I am looking to setup Artemis for my use-cases that are currently using
ActiveMQ v5.14.1 and want to use Kerberos auth which isn't available in
ActiveMQ currently. 
How can i set-up Kerberos authentication for openwire and STOMP protocol?
How will clients (STOMP and openwire) authenticate to the server?
The migration guide 
https://activemq.apache.org/artemis/migration/authentication.html
  
doesn't have a section about Kerberos authentication.

Is it possible to setup Kerberos auth with above protocols? Any examples or
docs on this would help.

Thanks,
Abhi



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: Security features in Artemis 2.x

[xabhi]

Thanks Christopher for the migration guide. It wasn't linked in the artemis
user manual. It is a useful resource.

As I mentioned Kerberos support would be a big motivation for an early
migration to Artemis. In the user manual it isn't clear to what extent
Kerberos is supported like it is mentioned only for amqp transport. 
Can I use Kerberos with tcp connector and STOMP protocol? Any docs or
examples for configuring this would be of great help.

Thanks,
Abhi



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: Security features in Artemis 2.x

[xabhi]

Hi,

Can any Artemis dev please comment on these? The documentation doesn't touch
upon these points and it isn't clear in what aspects the security is
different/better when compared to ActiveMQ

Thanks,
Abhishek



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Security features in Artemis 2.x

[xabhi]

Hi,

I am currently using ActiveMQ v5.14 and have been using it in production for
quite a long time without any issues. I want to use authentication and
authorization in ActiveMQ but haven't because of lack of Kerberos support
and have been following - https://issues.apache.org/jira/browse/AMQ-4693. I
am not sure when this will be available. 
Artemis documentation mentions Kerberos support in Security section but it
isn't clear to what extent it is supported. I mainly use Openwire and STOMP
protocol. Clients are JMS, .NET, Python/Perl/JS (STOMP)
- is it available for tcp connector?
- can i use it with STOMP protocol
- How will kerberos authentication work? Any examples for this
- how authorization will be done with respect to a user? How will Artemis
know which user has which role?

Other compatibility concerns are like:
- current clients being able to connect to Artemis without any issues. Is
there a migration guide for server/client?
-Multi-kahadb support
-Automatically destroy offline durable subscribers etc.

It would be really helpful to know what support from above is present in
current Artemis version and to what extent or if I should log any JIRA.
Security and Kerberos support would be a big motivation for me to migrate to
Artemis. I am really excited to see Artemis as ActiveMQ 6. 

Thanks,
Abhi





I use features like multi-kahadb, automatic destruction of offline durable
subscribers



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

/bin/activemq command not working

[xabhi]

Hi,
I am trying to get destination stats using /bin/activemq command. This was
earlier available via activemq-admin command. (
http://activemq.apache.org/activemq-command-line-tools-reference.html
  )

? ~/apache-activemq-5.14.4/bin/activemq dstat topics
-Dactivemq.jmx.url=service:jmx:rmi:///jndi/rmi://mqbrokerqa:9011/jmxrmi 
...
Connecting to JMX URL: service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi
INFO: Broker not available at:
service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi

It is not picking up the jmx url.

However, below command works:
./activemq query --view InactiveDurableTopicSubscribers --jmxurl
service:jmx:rmi:///jndi/rmi://mqbrokerqa:9011/jmxrmi

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/bin-activemq-command-not-working-tp4728517.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Switch to Artemis from ActiveMQ v5.14.4?

[xabhi]

How hard would be to switch just the messaging server from ActiveMQ to
Artemis. There are a lot of JMS, Perl, Python clients connecting to our
ActiveMQ instance.

Is Artemis being developed as drop-in replacement for ActiveMQ that can work
seamlessly with activemq-client libraries without much changes? It would be
great if I can use it as a drop-in replacement for my current ActiveMQ
instance without disturbing client side code, much easier to test if it is
possible.

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Switch-to-Artemis-from-ActiveMQ-v5-14-4-tp4727014p4727185.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Object messages in ActiveMQ v5.14.4

[xabhi]

Hi,

I am upgrading my ActiveMQ setup from v5.11.1 to v5.14.4. I came across this
wiki page as per upgrade instructions -
http://activemq.apache.org/objectmessage.html.

It says that i need to org.apache.activemq.SERIALIZABLE_PACKAGES system
property to whitelist the packages. However, it isn't clear that whether
this property is required only on Client side or if it is required on broker
side as well.

In my testing the Object messages were successfully sent from producer
client to consumer client by specifying the system property only on client
side and nothing on broker side. 

Can anyone confirm this behavior and whether the property should be added to
broker as well?

Thanks
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Object-messages-in-ActiveMQ-v5-14-4-tp4726341.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Unmatched acknowledge: MessageAckExpected - message count (1) differs from count in dispatched-list (2)

[xabhi]

Hi Tim,
Thanks for the reply. I already submitted this as a bug in JIRA, see 
https://issues.apache.org/jira/browse/AMQ-6622
  , but no one responded
there so I had to cross post on the forum.

I would like to know if there is any workaround for this bug in case it
happens in production? Is restart the only way to fix it?

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Unmatched-acknowledge-MessageAckExpected-message-count-1-differs-from-count-in-dispatched-list-2-tp4723664p4723739.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Unmatched acknowledge: MessageAckExpected - message count (1) differs from count in dispatched-list (2)

[xabhi]

Observed below exceptions in STOMP consumer after failover. The consumer
continued to receive messages but this exception kept coming on both client
and server side and when the master activemq instance took over after
another failover, all the messages were redelivered and below warning went
away.
I think it is a bug of some kind, can someone from ActiveMQ Dev team help
here. 

Exception:
[20170310 00:24:22:092 stomp_client.py:82 ERROR] Received an error:
org.apache.activemq.transport.stomp.ProtocolException: Unexpected ACK
received for message-id
[ID:diogenes31.nyc.deshaw.com-41001-1489085659582-1:4:-1:1:1]
at
org.apache.activemq.transport.stomp.ProtocolConverter.onStompAck(ProtocolConverter.java:475)
at
org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:250)
at
org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:85)
at
org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
at
org.apache.activemq.transport.stomp.StompCodec.processCommand(StompCodec.java:129)
at
org.apache.activemq.transport.stomp.StompCodec.parse(StompCodec.java:100)
at
org.apache.activemq.transport.stomp.StompNIOTransport.processBuffer(StompNIOTransport.java:136)
at
org.apache.activemq.transport.stomp.StompNIOTransport.serviceRead(StompNIOTransport.java:121)
at
org.apache.activemq.transport.stomp.StompNIOTransport.access$000(StompNIOTransport.java:44)
at
org.apache.activemq.transport.stomp.StompNIOTransport$1.onSelect(StompNIOTransport.java:73)
at
org.apache.activemq.transport.nio.SelectorSelection.onSelect(SelectorSelection.java:98)
at
org.apache.activemq.transport.nio.SelectorWorker$1.run(SelectorWorker.java:118)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
 
[20170310 00:24:22:096 stomp_client.py:82 ERROR] Received an error:
javax.jms.JMSException: Unmatched acknowledge: MessageAck
{commandId = 5, responseRequired = false, ackType = 2, consumerId =
ID:diogenes31.nyc.deshaw.com-41001-1489085659582-1:9:-1:1, firstMessageId =
null, lastMessageId =
ID:diogenes31.nyc.deshaw.com-41001-1489085659582-1:4:-1:1:2, destination =
topic://run1.topic.0, transactionId = null, messageCount = 1, poisonCause =
null}
; Expected message count (1) differs from count in dispatched-list (2)
at
org.apache.activemq.broker.region.PrefetchSubscription.assertAckMatchesDispatched(PrefetchSubscription.java:520)
at
org.apache.activemq.broker.region.PrefetchSubscription.acknowledge(PrefetchSubscription.java:212)
at
org.apache.activemq.broker.region.AbstractRegion.acknowledge(AbstractRegion.java:528)
at
org.apache.activemq.broker.region.RegionBroker.acknowledge(RegionBroker.java:484)
at
org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:88)
at
org.apache.activemq.broker.TransactionBroker.acknowledge(TransactionBroker.java:276)
at
org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:88)
at
org.apache.activemq.broker.MutableBrokerFilter.acknowledge(MutableBrokerFilter.java:98)
at
org.apache.activemq.broker.util.LoggingBrokerPlugin.acknowledge(LoggingBrokerPlugin.java:162)
at
org.apache.activemq.broker.MutableBrokerFilter.acknowledge(MutableBrokerFilter.java:98)
at
deshaw.tools.jms.ActiveMQLoggingPlugin.acknowledge(ActiveMQLoggingPlugin.java:453)
at
org.apache.activemq.broker.MutableBrokerFilter.acknowledge(MutableBrokerFilter.java:98)
at
org.apache.activemq.broker.TransportConnection.processMessageAck(TransportConnection.java:590)
at
org.apache.activemq.command.MessageAck.visit(MessageAck.java:245)
at
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:336)
at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:200)
at
org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:45)
at
org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:301)
at
org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:97)
at
org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:202)
at
org.apache.activemq.transport.stomp.ProtocolConverter.onStompAck(ProtocolConverter.java:456)
at
org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:250)
at

activemq-admin command line missing ActiveMQ v5.14.3

[xabhi]

Hi,

I see that ActiveMQ admin command 'bin/activemq-admin' is removed in
v5.14.3. I was unable to find the relevant ticket in JIRA. Could anyone
explain why it was removed and what is the alternative? I use it to collect
various usage statistics in my ActiveMQ setup which are very helpful. The
corresponding documentation (
http://activemq.apache.org/activemq-command-line-tools-reference.html
  )
doesn't say anything about removal. 

Is there any other command line tool similar to 'activemq-admin query' with
which i can get the usage statistics about destinations etc.

Thanks,
Abhi




--
View this message in context: 
http://activemq.2283324.n4.nabble.com/activemq-admin-command-line-missing-ActiveMQ-v5-14-3-tp4721561.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Durable subscription queries and ways to handle rogue durable subscribers

[xabhi]

Hi,
I wanted to know a few things about durable topic subscription which i
couldn't find on ActiveMQ documentation

1. Is it possible to purge all pending messages on the durable topic
subscription when the subscriber is not active, similar to purge operation
on the queues?

2. Is there a way to configure offlineDurableSubscriberTimeout on a per
topic basis? Currently my understanding is that it is a global setting (
http://activemq.apache.org/manage-durable-subscribers.html
  )

The problem I am facing is that I have a topic with a producer that produces
throughout the day with a quite a high number of messages and if the durable
subscriber goes down for more than a few hours then a lot of messages gets
piled up. This has caused issues several times like -  activemq instance
gets unresponsive due to high memory consumption, the kahadb gets corrupted
because we have manually delete kahadb files as destory method doesn't work
in this situation etc.

Is there any other strategy to handle above situation?

Thanks,
Abhi




--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Durable-subscription-queries-and-ways-to-handle-rogue-durable-subscribers-tp4721559.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Disable producer flow control in ActiveMQ v5.14.3

[xabhi]

Hi ActiveMQ Devs,

Is this a bug that producer is stopped even if the producer flow control is
disabled or is this expected?

Can anyone explain this behavior?

Thanks,
Abhishek



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Disable-producer-flow-control-in-ActiveMQ-v5-14-3-tp4720786p4721010.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Ignoring sub command log messages

[xabhi]

Thanks christopher for the explanation. This makes sense. What are the
timelines for ActiveMQ v5.15.0 release - mid year? I am asking because I
plan to upgrade to 5.14.3 and if this issue will be handled well in 5.15.0 i
would love to upgrade directly to 5.15.0.

Thanks,
Abhi





--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Ignoring-sub-command-log-messages-tp4720928p4720977.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Ignoring sub command log messages

[xabhi]

Hi,

Yes these log messages appeared at the startup only. I had faced an issue
where activemq dumped  7gb file - tx-14693130-1483907222896.tmp in one of
the mkahadb directories when i tried to destroy a durable topic consumer
which was inactive for quite some time and had hit the memory threshold of
70% (this was causing activemq to be unresponsive). I removed this file
manually and when failover switch happened I saw above log messages in log
file.

Is there a clean way to discard all the messages in durable topic
subscription without dumping a 7gb file?

Thanks,
Abhishek



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Ignoring-sub-command-log-messages-tp4720928p4720974.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Ignoring sub command log messages

[xabhi]

Hi,

I saw below log messages in my ActiveMQ instance's logs. When does this
happen and what effects can it have?

[20170108 15:29:19.584 EST (main)
org.apache.activemq.store.kahadb.MessageDatabase#updateIndex 1405 DEBUG] -
no message sequence exists for id:
ID:geneva13.nyc.43322-1482156880537-3:13196:1:1:3 and sub:
gear-munshi-geneva.gear.sync.listener.GelbaseGenevaMessageListener 
[20170108 15:29:19.584 EST (main)
org.apache.activemq.store.kahadb.MessageDatabase#updateIndex 1405 DEBUG] -
no message sequence exists for id:
ID:geneva13.nyc.43322-1482156880537-3:13196:1:1:4 and sub:
gear-munshi-geneva.gear.sync.listener.GelbaseGenevaMessageListener 
[20170108 15:29:19.584 EST (main)
org.apache.activemq.store.kahadb.MessageDatabase#updateIndex 1405 DEBUG] -
no message sequence exists for id:
ID:geneva13.nyc.43322-1482156880537-3:1:2:1:5946 and sub:
gear-munshi-listener:geneva.gear.sync.listener.GelbaseGenevaMessageListener 
[20170108 15:29:19.585 EST (main)
org.apache.activemq.store.kahadb.MessageDatabase#process 1047 DEBUG] -
ignoring add sub command during recovery replay:destination {
  type: TOPIC
  name: geneva.loader.messageTopic
}
subscriptionKey:
gear-munshi-listener:geneva.gear.sync.listener.GelbaseGenevaMessageListener
retroactive: false
subscriptionInfo: org.apache.activemq.protobuf.Buffer@ffc5
 
[20170108 15:29:19.585 EST (main)
org.apache.activemq.store.kahadb.MessageDatabase#process 1047 DEBUG] -
ignoring add sub command during recovery replay:destination {
  type: TOPIC
  name: gear.tdpBundleCompletionTopic
}
subscriptionKey:
post-tdp-bundle-listener:geneva.gear.sync.listener.PostTdpMessageListener
retroactive: false
subscriptionInfo: org.apache.activemq.protobuf.Buffer@ffea


Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Ignoring-sub-command-log-messages-tp4720928.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Unable to launch ActiveMQ v5.14.3 with Jetty

[xabhi]

Looks like it is due to Jetty upgrade in ActiveMQ version and
SelectChannelConnector seems to have been removed in new jetty version (
http://stackoverflow.com/questions/36788049/selectchannelconnector-alternative-class-in-jetty-9-and-above

 
)





--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Unable-to-launch-ActiveMQ-v5-14-3-with-Jetty-tp4720841p4720855.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Unable to launch ActiveMQ v5.14.3 with Jetty

[xabhi]

Below is the minimum configuration required to reproduce this problem. I am
using these files will freshly extracted ActiveMQ v5.14.3 with no changes.

I am not sure what is missing as this works fine with v5.11.1 with below
command:
~/apache-activemq-5.11.1/bin/activemq console
xbean:/u/choudhab/activemq/activemq-test/test-broker.xml

*Command used to run:*
~/apache-activemq-5.14.3/bin/activemq console
xbean:/u/choudhab/activemq/activemq-test/test-broker.xml


*Broker config file:*


http://www.springframework.org/schema/beans;
xmlns:amq="http://activemq.apache.org/schema/core;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation="http://www.springframework.org/schema/beans
   
classpath:org/springframework/beans/factory/xml/spring-beans-2.0.xsd
http://activemq.apache.org/schema/core
classpath:activemq.xsd">





















   




  







*Jetty config file:*


http://www.springframework.org/schema/beans;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd;>

























































--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Unable-to-launch-ActiveMQ-v5-14-3-with-Jetty-tp4720841p4720848.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Unable to launch ActiveMQ v5.14.3 with Jetty

[xabhi]

Hi,
I am trying to use new version of ActiveMQ v5.14.3 but i am not able to
launch it with embedded jetty server of AJAX. The same setup is working for
ActiveMQ v5.11.1 without any issues:

The jetty version in 5.11.1 is v8.1.16.v20140903.

*Java command for v5.14.3:*
Java Runtime: Oracle Corporation 1.8.0_60
/usr/local/java/jdk1.8.0.60-2016i/jre
  Heap sizes: current=2010112k  free=1968127k  max=7456768k
JVM args: -Xmx8g -XX:ErrorFile=/var/tmp/hs_err_pid%p.log
-Dactivemq.broker.name=mqbrokertest -Dactivemq.broker.type=primary
-Dactivemq.classpath=/proj/tools/build/stable/base//java/jar/common.jar:/proj/tools/build/stable/base//java/jar/sonar.jar:/proj/tools/build/stable/base//java/jar/tools.jar
-D
activemq.data.dir=/u/amqtest/test//kahadb/ -Dactivemq.jmx.port=9006
-Dactivemq.log.dir=/u/amqtest/test//logs/
-Dactivemq.webapps.dir=/u/amqtest/test//webapps/
 -Djava.library.path=/proj/tools/build/stable/base//lib/Linux.RHEL7.x86_64
-Dlog4j.configuration=
file:/proj/tools/build/stable/base//etc/config/activemq/log4j.properties
-Dorg.apache.activemq.audit=true
-Dorg.apache.activemq.store.kahadb.LOG_SLOW_ACCESS_T
IME=5000
Extensions classpath:
 
[/proj/tools/infra/apache/activemq/apache-activemq-5.14.3/lib,/proj/tools/infra/apache/activemq/apache-activemq-5.14.3/lib/camel,/proj/tools/infra/apache/ac
tivemq/apache-activemq-5.14.3/lib/optional,/proj/tools/infra/apache/activemq/apache-activemq-5.14.3/lib/web,/proj/tools/infra/apache/activemq/apache-activemq-
5.14.3/lib/extra]
ACTIVEMQ_HOME: /proj/tools/infra/apache/activemq/apache-activemq-5.14.3
ACTIVEMQ_BASE: /proj/tools/infra/apache/activemq/apache-activemq-5.14.3
ACTIVEMQ_CONF: /proj/tools/infra/apache/activemq/apache-activemq-5.14.3/conf
ACTIVEMQ_DATA: /proj/tools/infra/apache/activemq/apache-activemq-5.14.3/data
Loading message broker from:
xbean:file:/u/amqtest/config/amq_test_broker_config.xml


I get below class not found exception but i haven't changed anything except
the activemq version:

ERROR: java.lang.RuntimeException: Failed to execute start task. Reason:
org.springframework.beans.factory.BeanCreationException: Error creating bean
with nam
e 'Server' defined in URL
[file:/proj/tools/build/stable/base/etc/config/activemq/amq_jetty_config.xml]:
Cannot create inner bean 'Connector' of type [org.ecl
ipse.jetty.server.nio.SelectChannelConnector] while setting bean property
'connectors' with key [0]; nested exception is
org.springframework.beans.factory.Can
notLoadBeanClassException: Cannot find class
[org.eclipse.jetty.server.nio.SelectChannelConnector] for bean with name
'Connector' defined in URL [file:/proj/t
ools/build/stable/base/etc/config/activemq/amq_jetty_config.xml]; nested
exception is java.lang.ClassNotFoundException:
org.eclipse.jetty.server.nio.SelectCha
nnelConnector
java.lang.RuntimeException: Failed to execute start task. Reason:
org.springframework.beans.factory.BeanCreationException: Error creating bean
with name 'Serv
er' defined in URL
[file:/proj/tools/build/stable/base/etc/config/activemq/amq_jetty_config.xml]:
Cannot create inner bean 'Connector' of type [org.eclipse.je
tty.server.nio.SelectChannelConnector] while setting bean property
'connectors' with key [0]; nested exception is
org.springframework.beans.factory.CannotLoad
BeanClassException: Cannot find class
[org.eclipse.jetty.server.nio.SelectChannelConnector] for bean with name
'Connector' defined in URL [file:/proj/tools/bu
ild/stable/base/etc/config/activemq/amq_jetty_config.xml]; nested exception
is java.lang.ClassNotFoundException:
org.eclipse.jetty.server.nio.SelectChannelCon
nector
at
org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:91)
at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)
at
org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:154)
at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)
at
org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
at org.apache.activemq.console.Main.main(Main.java:115)
Caused by: org.springframework.beans.factory.BeanCreationException: Error
creating bean with name 'Server' defined in URL
[file:/proj/tools/build/stable/base/etc/config/activemq/amq_jetty_config.xml]:
Cannot create inner bean 'Connector' of type
[org.eclipse.jetty.server.nio.SelectChannelConnector] while setting bean
property 'connectors' with key [0]; nested exception is
org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find
class 

Disable producer flow control in ActiveMQ v5.14.3

[xabhi]

Hi, 
I am trying out new ActiveMQ v5.14.3 and observer below producer flow
control logs in my setup even though I have disabled producer flow control
in the broker config file. Is there something missing in the config file or
has anything changed in the new version

Could anyone explain this behavior?

Logs:
[20170102 07:32:58.897 EST (ActiveMQ NIO Worker 2)
org.apache.activemq.broker.region.BaseDestination#waitForSpace 693 INFO] -
Usage(default:temp:queue://seed.tcaHeartBeat:temp) percentUsage=0%,
usage=5368709290, limit=5368709120,
percentUsageMinDelta=1%;Parent:Usage(default:temp) percentUsage=100%,
usage=5368709290, limit=5368709120, percentUsageMinDelta=1%: Temp Store is
Full (0% of 5368709120). Stopping producer
(ID:socrates45.nyc-43058-1483064654695-3:1:2:1) to prevent flooding
queue://seed.tcaHeartBeat. See
http://activemq.apache.org/producer-flow-control.html for more info
(blocking for: 2703s) 

Config file:












Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Disable-producer-flow-control-in-ActiveMQ-v5-14-3-tp4720786.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

'Client already connected ' exception in ActiveMQ v5.11.1

[xabhi]

Hi,

Setup:
ActiveMQ v5.11.1
Clients connect to ActiveMQ server via load balancer
Client is a durable stomp subscriber on a topic

I have come across a weird case where I am getting a 'client already
connected from some ip' exception but there is not client running and the
durable subscriber is marked as Active.
There is not record of the concerned client's 'client-id' and
'remoteAddress' in jconsole under "clientConnectors" tab.

I have received this exception earlier also with some genuine cases. But
this time I am sure that there is no other client running but i am still
getting this exception.

This looks like an ActiveMQ bug. Does anyone has any idea, how to handle it
- since there is no way to kill a client thats non-existent and it is a
'production' setup where i cannot do casual restarts.

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Client-already-connected-exception-in-ActiveMQ-v5-11-1-tp4715869.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ArrayIndexOutOfBoundsException in ActiveMQ v5.11.1

[xabhi]

Hi Tim,

Thanks for taking a look at this. But rural hunter encountered this problem
in ActiveMQ v5.13.3 and I am seeing these warnings on server in ActiveMQ
v5.11.1

Also do you have any idea what can cause ArrayIndexOutOfBounds exception on
client side in ActiveMQ v5.11.1 ( see first message

 
) ? Is this is a bug?

Could you give some pointers that can help identify the cause of these
warnings so that I can debug this at my end.

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/ArrayIndexOutOfBoundsException-in-ActiveMQ-v5-11-1-tp4715139p4715698.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ArrayIndexOutOfBoundsException in ActiveMQ v5.11.1

[xabhi]

Hi,

What do these exceptions indicate? I saw below exceptions along with the
ArrayIndexOutOfBounds exception but without the stack strace i an not able
to figure out anything?

Has anyone else encountered these exceptions before?

Exceptions:
org.apache.activemq.broker.TransportConnection#serviceTransportException 245
WARN] - Transport Connection to: tcp://10.79.30.101:33184 failed:
java.io.IOException: Direct buffer memory : 

org.apache.activemq.broker.TransportConnection#serviceTransportException 245
WARN] - Transport Connection to: tcp://10.79.27.107:29377 failed:
java.io.IOException: Java heap space : 

Please help as I am encountering these exceptions almost daily in
production? The log messages are not very informative about what has to be
done as well.

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/ArrayIndexOutOfBoundsException-in-ActiveMQ-v5-11-1-tp4715139p4715293.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ArrayIndexOutOfBoundsException in ActiveMQ v5.11.1

[xabhi]

Is this something I should worry about happening again? What scenarios can
cause the ArrayIndexOutOfBoundsException exception (in the first post)?

Thanks,
Abhi





--
View this message in context: 
http://activemq.2283324.n4.nabble.com/ArrayIndexOutOfBoundsException-in-ActiveMQ-v5-11-1-tp4715139p4715253.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

ArrayIndexOutOfBoundsException in ActiveMQ v5.11.1

[xabhi]

Hi ActiveMQ devs,

I observed below ArrayIndexOutOfBounds exception in my production setup and
it caused high non-persistent memory usage on the affected topic due to
transaction rollbacks. It is not clear why this happened.

Could you please help in investigating this? What could be the possible
causes for this?

[ 2016-08-08 06:37:34,321 [ActiveMQ Transport:
tcp://mqbrokerprod.deshaw.com/149.77.3.169:61616@52260] 
failover.FailoverTransport.handleTransportFailure():260 WARN ]: Transport
(tcp://mqbrokerprod.deshaw.com/149.77.3.169:61616@52260) failed, reason:  ,
attempting to automatically reconnect
java.io.IOException: Unexpected error occured:
java.lang.ArrayIndexOutOfBoundsException: 12851
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:203)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 12851
at
org.apache.activemq.openwire.OpenWireFormat.getFromUnmarshallCache(OpenWireFormat.java:534)
at
org.apache.activemq.openwire.v10.BaseDataStreamMarshaller.tightUnmarsalCachedObject(BaseDataStreamMarshaller.java:148)
at
org.apache.activemq.openwire.v10.MessageMarshaller.tightUnmarshal(MessageMarshaller.java:75)
at
org.apache.activemq.openwire.v10.ActiveMQMessageMarshaller.tightUnmarshal(ActiveMQMessageMarshaller.java:66)
at
org.apache.activemq.openwire.v10.ActiveMQTextMessageMarshaller.tightUnmarshal(ActiveMQTextMessageMarshaller.java:66)
at
org.apache.activemq.openwire.OpenWireFormat.tightUnmarshalNestedObject(OpenWireFormat.java:445)
at
org.apache.activemq.openwire.v10.BaseDataStreamMarshaller.tightUnmarsalNestedObject(BaseDataStreamMarshaller.java:125)
at
org.apache.activemq.openwire.v10.MessageDispatchMarshaller.tightUnmarshal(MessageDispatchMarshaller.java:71)
at
org.apache.activemq.openwire.OpenWireFormat.doUnmarshal(OpenWireFormat.java:354)
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:268)
at
org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:221)
at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:213)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
... 1 more

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/ArrayIndexOutOfBoundsException-in-ActiveMQ-v5-11-1-tp4715139.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Is there a way to disconnect an active durable topic subscriber

[xabhi]

Hi,

I recently encountered a problematic scenario where a stray process was
running with a client-id in the cluster and it prevented start of another
client with same client-id. I wasn't able to find the host of the stray
process so as a temporary fix i started the new process with a different
client-id and it worked.

I know that we can destroy inactive durable subscriptions from jconsole but
i wasn't able to do it for this case. Is there a way to disconnect/destroy
an active durable subscription?

This will help in resolving such issues or even in case of problematic
clients where we don't have any control on the client process (say the
process is sending large messages which is filling up kahadb partition).

Thanks!
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Is-there-a-way-to-disconnect-an-active-durable-topic-subscriber-tp4713970.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Producer slows down when consumer is slow

[xabhi]

Hi,

Below is my destination policy in which I have disabled producerFlowControl
but my producer still slows down (to the rate consumer is able to actually
handle).
Why does this happen even after disabling producerFlowControl, are these two
things not related?? What should i do to avoid this?

amq:destinationPolicy>










Thanks,
Abhi





--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Producer-slows-down-when-consumer-is-slow-tp4712723.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Message Redelivery in STOMP client using NACK

[xabhi]

Hi,

Will ActiveMQ redeliver the message for which the STOMP client sent a NACK
frame?

I found this ActiveMQ page(
http://activemq.apache.org/how-do-i-unack-the-message-with-stomp.html
  )
but not sure that it explains the behavior correctly since we have NACK in
STOMP protocol to say that message was not consumed.

Could someone from ActiveMQ dev confirm the exact behavior as the STOMP
specification also says that the behavior is server specific and I didn't
find any documentation in ActiveMQ wiki

Thanks,
Abhishek



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Message-Redelivery-in-STOMP-client-using-NACK-tp4710711.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: JMS to STOMP transformation causes throughput drop in STOMP consumers

[xabhi]

Hi,

I am seeing the same behavior with Python STOMP consumer as well -
throughput for TEXT msgs is 10K/s and for MAP messages with tranformation is
200 msgs/s.

 Is there a way to improve it for JMS Map messages? Some broker setting? may
be plugin different serialization library instead of xstream?

Thanks,
Abhishek




--
View this message in context: 
http://activemq.2283324.n4.nabble.com/JMS-to-STOMP-transformation-causes-throughput-drop-in-STOMP-consumers-tp4710148p4710409.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ Object Message to json transformation not working

[xabhi]

A little unrelated question but does using 'transformation' header on STOMP
consumer causes throughput drop?

I tested it with jms-map-json transformation and throughput dropped to 200
msgs/s when compared to that simple text messages.

Should i do transformation to text message in my Java application? What will
i be losing if i do that because throughput is real concern for me.

I have asked this question here: 
http://activemq.2283324.n4.nabble.com/JMS-to-STOMP-transformation-causes-throughput-drop-in-STOMP-consumers-td4710148.html

  

Sorry for asking this question here but nobody replied there.

Any pointers on what could be causing it would help me greatly. Thanks for
all the help.



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/ActiveMQ-Object-Message-to-json-transformation-not-working-tp4709447p4710243.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

JMS to STOMP transformation causes throughput drop in STOMP consumers

[xabhi]

Hi,
I am trying to benchmark throughput for my nodejs consumer (STOMP). The
producer is in Java sending JMS text and map messages.

With text messages, I see that nodejs consumer is able to handle 10
Kmsgs/sec without any pending messages

But when I send Map messages and using nodejs consumer (with header
'transformation': jms-map-json), the throughput drops to 0.5 Kmsgs/sec.

I am not able to understand where this bottleneck is coming. The broker has
messages in pending queue and i see unacknowledged messages in jconsole.

Why is that the node consumer can consumer text messages faster that map
messages if ultimately both are sent in TEXT format from ActiveMQ?

Does anyone from ActiveMQ dev knows about this behavior? Any help will be
appreciated

Thanks,
Abhishek




--
View this message in context: 
http://activemq.2283324.n4.nabble.com/JMS-to-STOMP-transformation-causes-throughput-drop-in-STOMP-consumers-tp4710148.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ Object Message to json transformation not working

[xabhi]

How do I make sure that this class is available at broker side?



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/ActiveMQ-Object-Message-to-json-transformation-not-working-tp4709447p4710145.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

ActiveMQ Object Message to json transformation not working

[xabhi]

Hi,

I am having trouble sending a custom Object Message to a STOMP consumer with
ActiveMQ v5.11.1

In Jconsole, I can see messages being dequeued by STOMP consumer but the
subscription callback is never called.

However note that if I send a simple string as object message it gets
processed by STOMP consumer and I receive a JSON formatted message in this
format:
{"string":"hello"}

I create the message as follows:
myMessage = mySession.createObjectMessage("hello");
myProducers.get(destName).send(myMessage);

I am not sure what am i doing wrong here. Could anyone tell what is wrong
with this code?



Object Message class:
public static class SampleObjectMessage implements Serializable
{
private static final long serialVersionUID = 1L;
public String data;
public long sendTimeMs;
public SampleObjectMessage(String data, long sendTimeMs) {
this.data = data;
this.sendTimeMs = sendTimeMs;
}
public String getData() {
return data;
}
public void setData(String data) {
this.data = data;
}
public long getSendTimeMs() {
return sendTimeMs;
}
public void setSendTimeMs(long sendTimeMs) {
this.sendTimeMs = sendTimeMs;
}
}


Code for sending ObjectMessage:
myMessage = mySession.createObjectMessage(
 new SampleObjectMessage(getMessageText(ourMessageSize),
 System.currentTimeMillis()));

myProducers.get(destName).send(myMessage);



STOMP consumer has following headers for subscription:

var subscriptionHeaders = {
  'ack' : 'auto',
  'transformation' : 'jms-object-json'
};






--
View this message in context: 
http://activemq.2283324.n4.nabble.com/ActiveMQ-Object-Message-to-json-transformation-not-working-tp4709447.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Node.js STOMP client recommendation

[xabhi]

Hi,

I am looking at these two libraries -  https://www.npmjs.com/package/stompit
   and 
https://www.npmjs.com/package/stompjs
  . 

Has anyone used them before? Which one is recommended by ActiveMQ community?





--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Node-js-and-ActiveMQ-communication-tp4707922p4708344.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Node.js and ActiveMQ communication

[xabhi]

Has anyone used JavaScript Stomp clients with ActiveMQ? Which Client library
do ActiveMQ developers recommend?

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Node-js-and-ActiveMQ-communication-tp4707922p4708061.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Node.js and ActiveMQ communication

[xabhi]

Hi,

I want to send messages between ActiveMQ and node server. Which STOMP client
library is recommended?
I see there are two listed on  http://activemq.apache.org/websockets.html
  . 

Does these libraries implement features like acknowledgements, receipts,
heart-beating, auto-failover between brokers etc.?

I see that  stomp-websocket    is
not being maintained currently? Should I consider using this?

Which is more preferred by ActiveMQ community - Stomple or Stomp.js for use
with Node.js?

Thanks,
Abhishek



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Node-js-and-ActiveMQ-communication-tp4707922.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Understanding ActiveMQ Inactivity timeout monitor in STOMP

[xabhi]

I am using ActiveMQ v5.11.1

Below are the STOMP frames sent/received from client/server.

# Net::STOMP::Client->connect()
#  encoded CONNECT frame
#   H client-id:test_perl_consumer_huygens01-test_stomp_consumer
#   H passcode:
#   H host:mqbrokerdev
#   H accept-version:1.0,1.1
#   H login:
#   sent 141 bytes
#   received 135 bytes
#  decoding CONNECTED frame
#   H server:ActiveMQ/5.11.1
#   H heart-beat:0,0
#   H session:test_perl_consumer_huygens01-test_stomp_consumer
#   H version:1.1

# Net::STOMP::Client->subscribe()
#  encoded SUBSCRIBE frame
#   H activemq.prefetchSize:1
#   H
activemq.subscriptionName:test_perl_consumer_huygens01-test_stomp_consumer
#   H ack:client
#   H destination:/topic/test
#   H callback:CODE(0x33a1188)
#   H id:/topic/test-huygens01-15146-stomp_consumer.pl
#   sent 254 bytes
Subscribed to /topic/test

Thanks,
Abhi




--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Understanding-ActiveMQ-Inactivity-timeout-monitor-in-STOMP-tp4704227p4704280.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Understanding ActiveMQ Inactivity timeout monitor in STOMP

[xabhi]

Hi,

Below is my ActiveMQ configuration file. I have specified
wireFormat.maxInactivityDuration for TCP Nio transport. Does this inactivity
timeout applies to STOMP connector as well?

I am seeing cases where if my STOMP client is busy for more than 5 mins in
doing some operation and doesn't send heartbeats, ActiveMQ closes that
connection. Does broker logs anything for indicating this timeout?
Another confusing thing is why does timeout happens only after 5 mins, if it
uses same inactivity value the timeout should happen after 2 mins, right?

Could anyone please explain this behavior?



wireFormat.maxInactivityDuration=12&*amp;maximumConnections=600wireFormat.maxFrameSize=157286400"
   
maximumProducersAllowedPerConnection="200"
   
maximumConsumersAllowedPerConnection="200" />
 




Thanks,
Abhi




--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Understanding-ActiveMQ-Inactivity-timeout-monitor-in-STOMP-tp4704227.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: How to get number of unacknowledged messages on the broker?

[xabhi]

Hi,

I am not sure what to do when you say using JMX to iterate over each
subscription, Where will you do this inside a broker plugin? do you mean to
create jmx connection and get those stats?
Can we not access those stats directly inside a broker plugin?

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/How-to-get-number-of-unacknowledged-messages-on-the-broker-tp4697616p4697658.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: No of consumers per connection in ActiveMQ

[xabhi]

Hi ActiveMQ gods,

Another ping!

Is there any suggestion on what is a good estimate for the number of
consumers/producers on a single connection?

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/No-of-consumers-per-connection-in-ActiveMQ-tp4696635p4696903.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: No of consumers per connection in ActiveMQ

[xabhi]

Hi,
Could anyone provide details on what should be the number of
consumers/producers on a single connection?

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/No-of-consumers-per-connection-in-ActiveMQ-tp4696635p4696716.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

No of consumers per connection in ActiveMQ

[xabhi]

Hi,
I recently observed that some of the clients in my ActiveMQ setup are having
around 100 consumers on the same connection. 
I wanted to check if it is okay to have this many consumers on same
connection? or I should split them across multiple connections?

Which is a more costly resource for ActiveMQ to maintain 1 more connection
or 100s of consumers on 1 connection?
What would be a good number for number of consumers on a single connection?

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/No-of-consumers-per-connection-in-ActiveMQ-tp4696635.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: KahabDB compatibility issue between ActiveMQ 5.10 and ActiveMQ 5.11

[xabhi]

I took some more time out and found that this inconsistency was not because
of any change in KahaDB but because of change in physicalName of
AnyDestination from 0 to !0. 

The commit refers to this JIRA issue: 
https://issues.apache.org/jira/browse/AMQ-5450
https://issues.apache.org/jira/browse/AMQ-5450  

Hope this helps.

Thanks,
Abhi





--
View this message in context: 
http://activemq.2283324.n4.nabble.com/KahabDB-compatibility-issue-between-ActiveMQ-5-10-and-ActiveMQ-5-11-tp4692008p4696611.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: KahabDB compatibility issue between ActiveMQ 5.10 and ActiveMQ 5.11

[xabhi]

Hi Tim,

Did you got a chance to look at this?

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/KahabDB-compatibility-issue-between-ActiveMQ-5-10-and-ActiveMQ-5-11-tp4692008p4696555.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: KahabDB compatibility issue between ActiveMQ 5.10 and ActiveMQ 5.11

[xabhi]

Hi Tim, 
I don't have any specific idea as to what changed between the versions but i
can point you to the part of code that does it.

In my mulitKahaDB adapater configuration I have this entry:

amq:filteredKahaDB
amq:persistenceAdapter
amq:kahaDB/
/amq:persistenceAdapter
/amq:filteredKahaDB

Which essentially defines persistence adapter for all destinations, which
corresponds to matchAll destination in MultiKahaDBPersistenceAdapter class:

final static ActiveMQDestination matchAll = new AnyDestination(new
ActiveMQDestination[]{new ActiveMQQueue(), new ActiveMQTopic()});

This is the call
IOHelper.toFileSystemSafeName(destination.getQualifiedName()); in
nameFromDestinationFilter() function returns the directory name for the
kahadb adapter.

This call earlier used to return 0 and now it returns #210.

You can run below two lines of code to check this with different versions of
ActiveMQ:

 final ActiveMQDestination matchAll =
   new AnyDestination(new ActiveMQDestination[]{new
ActiveMQQueue(), new ActiveMQTopic()});
   
System.out.println(IOHelper.toFileSystemSafeName(matchAll.getQualifiedName()));


Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/KahabDB-compatibility-issue-between-ActiveMQ-5-10-and-ActiveMQ-5-11-tp4692008p4696226.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: DOS attack on activemq setup

[xabhi]

Humble ping to get some help in completing this effort.

Another thing- The ampq protocol doesn't respect either of the
configurations for message size - transport.maxDataLength and
wireFormat.maxFrameSize.

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/DOS-attack-on-activemq-setup-tp4694598p4695975.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: DOS attack on activemq setup

[xabhi]

Hi,
I am having some difficulties/concerns in implementing some of the ideas:

1. Restricting composite destinations for producer and consumer:
I tried adding destination.iscomposite() checks in addProducer and
addConsumer by extending BrokerPluginSupport. The problem I am facing now is
that it is restricting addition of consumer on a composite destination
ActiveMQ.Advisory.TempQueue,ActiveMQ.Advisory.TempTopic which is added for
every producer and consumer connecting to broker and hence no
producer/consumer can connect If I use this strategy.
What would be other good way to achieve this.

Other concern is that I have dynamic producers also which can send messages
on composite destination at runtime. I am thinking of restricting this in
send call itself? Is it a good place to do this check?

2. Restricting size of messages sent to ActiveMQ:
I am using wireFormat.maxFrameSize to limit the size. But it works only for
clients using openwire. I have perl/python/php clients using STOMP protocol.
How to achieve this? Do I have make changes in client libraries? Or there is
some other configuration?

3. Restricting total topics and queues count:
I am trying to implement this in addDestination function. I am checking the
existence of destination being added in
getBrokerService().getRegionBroker().getDestinationMap() and if its new then
I check the count using getBrokerService().getAdminView(). Is it good to
expose DestinationMap in a plugin like this? Though I am not changing
anything here just a contains check.

Thanks,
Abhi




--
View this message in context: 
http://activemq.2283324.n4.nabble.com/DOS-attack-on-activemq-setup-tp4694598p4695749.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: KahabDB compatibility issue between ActiveMQ 5.10 and ActiveMQ 5.11

[xabhi]

ActiveMQ has changed the name of the default persistent adapter from 0 to
#210 in ActiveMQ 5.11.1.
ActiveMQ v5.11.1 will not read messages from 0 directory as there is no
persistence adapter defined for that and hence they are kind of lost.

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/KahabDB-compatibility-issue-between-ActiveMQ-5-10-and-ActiveMQ-5-11-tp4692008p4695679.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Limit the size of the message in ActiveMQ

[xabhi]

Hi,
I want to limit the size of the message that is sent to ActiveMQ. I have
clients in Perl, python, Java, C# and php.
What is the best way to put this limit? I know about
wireFormat.maxFrameSize. Does frame size accounts for entire frame or the
message payload only?
Will this be applicable to all  the protocols? I couldn't find any other
option related to message size.

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Limit-the-size-of-the-message-in-ActiveMQ-tp4695519.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Does the NIO connector use dramatically less memory?

[xabhi]

Hi,
I have also just started using NIO in my setup. The average memory usage is
same what it used to be with TCP but threads usage reduced a lot. Another
thing is I observe that memory usage oscillates a lot in case of NIO. I am
not sure if its a good thing ? May be someone from ActiveMQ dev team can
opine on this

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Does-the-NIO-connector-use-dramatically-less-memory-tp4695225p4695494.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: DOS attack on activemq setup

[xabhi]

Thanks gary!

I have logged a JIRA request for this enhancement - 
https://issues.apache.org/jira/browse/AMQ-5720
https://issues.apache.org/jira/browse/AMQ-5720  



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/DOS-attack-on-activemq-setup-tp4694598p4694867.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: DOS attack on activemq setup

[xabhi]

Hi gary,
In  http://activemq.apache.org/per-destination-policies.html
http://activemq.apache.org/per-destination-policies.html   the
*memoryLimit* option refers to only memoryUsage part of systemUsage as per
my understanding and there is no way to configure the storeUsage compoment
as of now. Just want to confirm my understanding here before I start using
this option.

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/DOS-attack-on-activemq-setup-tp4694598p4694869.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: DOS attack on activemq setup

[xabhi]

Hi gary,

In this reply 
http://activemq.2283324.n4.nabble.com/DOS-attack-on-activemq-setup-tp4694598p4694602.html
http://activemq.2283324.n4.nabble.com/DOS-attack-on-activemq-setup-tp4694598p4694602.html
  
you mentioned about controlling composite destinations. What kind of
problems can be caused using composite destinations from client side? I have
never used them and its unclear to me in what way it can abused to disrupt
broker service.








--
View this message in context: 
http://activemq.2283324.n4.nabble.com/DOS-attack-on-activemq-setup-tp4694598p4694883.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: DOS attack on activemq setup

[xabhi]

I just thought of another case where broker can be crashed. This is a case
where a rogue app sends copious amount of persistent messages with no
consumer and causes disk space to fill up making the broker unavailable.

I am using multiKahaDB in my setup and have grouped it based on destination
patterns (a logical grouping of related destinations). But the kind of
scenario I have described can be prevented from affecting other applications
If I can put size limit on each of multiKahaDB persistent adapter separately
otherwise one multiKahaDB can use entire remaining space for itself. That
way broker won't run out of space for all destinations but only a group of
destinations thus preventing  the crash of entire setup.

Is there any KahaDB configuration already present which will help in this
case?

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/DOS-attack-on-activemq-setup-tp4694598p4694820.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: DOS attack on activemq setup

[xabhi]

Hi Tim, rajdavies 
Could you please point me to documentation on how to restrict no. of
consumers/producers on transport connector. I couldn't find them on activemq
wiki.

All i could find was this page 
http://activemq.apache.org/tcp-transport-reference.html
http://activemq.apache.org/tcp-transport-reference.html  

Thanks
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/DOS-attack-on-activemq-setup-tp4694598p4694782.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

DOS attack on activemq setup

[xabhi]

Hi,

I was thinking about ways in which I cause DOS attack on activemq and how to
prevent it.

I can bring the setup down by:
1. creating large number of connections - restrict based on connectionID?
2. large number of destinations
3. large number of subscriptions, consumers, producers, wildcard
subscriptions etc - restrict wildcard subscription, limit no of
consumer/producer?
4. Sending large number of persistent/non-persistent messages with huge
sizes - limit msgsize that can be sent?

I don't know how to implement each of them and would like to get ActiveMQ
community's thought on how to prevent these scenarios (either by hacking
into/enriching activemq code - Plugins ?). What are other ways to create a
DOS attack on activemq?

I know ActiveMQ provides basic authentication/authorization
(username/password) to restrict some of these cases like authorization
policy for destinations based on user name, groups.

What I am talking about is an unintentional DOS attack- where an legitimate
application/client goes berserk to bug in code etc. and creates large number
of connections or does a wildcard subscription and start receiving all
messages etc.

I would like to get thought on how to prevent each of the cases I pointed
before.

Thanks,
Abhi




--
View this message in context: 
http://activemq.2283324.n4.nabble.com/DOS-attack-on-activemq-setup-tp4694598.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ 5.11.1 and STOMP v1.2

[xabhi]

Hi,
Just want to confirm if the information on this page 
https://stomp.github.io/implementations.html
https://stomp.github.io/implementations.html   is outdated?

Thanks,
Abhi




--
View this message in context: 
http://activemq.2283324.n4.nabble.com/ActiveMQ-5-11-1-and-STOMP-v1-2-tp4693847p4694048.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

ActiveMQ 5.11.1 and STOMP v1.2

[xabhi]

Hi,
Does ActiveMQ v5.11.1 implements STOMP v1.2 spec?

How can I check this?

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/ActiveMQ-5-11-1-and-STOMP-v1-2-tp4693847.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Incorrect inflight count for topic subscription

[xabhi]

So, What does inflight and queue size mean for a topic or queue? What is the
difference?

Should I ignore the topic stats completely? How do I know which ones are
accurate? I don't see any documentation on this?

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Incorrect-inflight-count-for-topic-subscription-tp4693602p4693645.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Clarification regarding broker statistics in jconsole

[xabhi]

Ping!
Please clarify this confusion regarding broker stats.

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Clarification-regarding-broker-statistics-in-jconsole-tp4693403p4693591.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Clarification regarding broker statistics in jconsole

[xabhi]

Hi, 
I have some confusion regarding the broker stats that are visible in
jconsole.

Why TotalConsumerCount != TopicSubscribers +
Temporary(topic/queue)Subscribers + QueueSubscribers +
DurableTopicSubscribers (excluding InactiveDurableSubscriber)
What should it be equal to in terms of subscriptions?

How does the total consumer count calculated? During addConsumer function?

TotalProducerCount in jconsole is shown as zero but dynamic producer count
is 243?

Thanks,
Abhishek



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Clarification-regarding-broker-statistics-in-jconsole-tp4693403.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: KahabDB compatibility issue between ActiveMQ 5.10 and ActiveMQ 5.11

[xabhi]

Ping. Can some one please explain any changes in kahadb when moving from
ActiveMQ 5.10 to ActiveMQ 5.11?





--
View this message in context: 
http://activemq.2283324.n4.nabble.com/KahabDB-compatibility-issue-between-ActiveMQ-5-10-and-ActiveMQ-5-11-tp4692008p4692323.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ 5.10.0 queue slowed down, restart helped

[xabhi]

Hi Tim,
The reason I think it might be related (though I use KahaDB for persistence)
is because this issue affected specifically the persistent messages in my
case, the issue happens every one week or two.
Restarting helped in consumption of messages.
In my case the messages consumption was delayed multiple times for some 30
mins or so ad some times for about an hour.
The resource consumption is also normal, nothing wrong with it
I haven't been able to reproduce it.

I am asking for pointers so that i can debug this issue better next time it
happens. For this same reason, I asked @mo to take a look at my observations
and see if they happens in their setup also.

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/ActiveMQ-5-10-0-queue-slowed-down-restart-helped-tp4690706p4691945.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ 5.10.0 queue slowed down, restart helped

[xabhi]

Hi,
I am also kind of facing a similar issue where ActiveMQ seems not be
delivering any messages to queue consumers (only for persistent messages
though). Consumers are not slow and there aren't any fast producers also.

http://activemq.2283324.n4.nabble.com/Broker-not-delivering-persistent-messages-to-consumer-on-queue-td4691245.html
http://activemq.2283324.n4.nabble.com/Broker-not-delivering-persistent-messages-to-consumer-on-queue-td4691245.html
  
 and 
http://stackoverflow.com/questions/28556049/activemq-not-delivering-dispatching-persistent-messages-on-queues?noredirect=1#comment45424344_28556049
http://stackoverflow.com/questions/28556049/activemq-not-delivering-dispatching-persistent-messages-on-queues?noredirect=1#comment45424344_28556049
  

I have posted this on SO also because nobody replied on the previous post.
Please checkout the links and see if any of the observations relates to you
situation.

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/ActiveMQ-5-10-0-queue-slowed-down-restart-helped-tp4690706p4691704.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Broker not delivering persistent messages to consumer on queue

[xabhi]

Can anyone suggest what configuration would be causing the broker to go in
this state where it is delivering non-persistent messages but not persistent
messages on queues?

I tried sending persistent messages from STOMP producers but those were also
not delivered.

This issue is not happening for topics. I am using ActiveMQ 5.10.0

This issue happens every weekend around same time.

Please suggest some direction/configuration I should look into.

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Broker-not-delivering-persistent-messages-to-consumer-on-queue-tp4691245p4691747.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Broker not delivering persistent messages to consumer on queue

[xabhi]

Can someone from activemq dev take a look at this and help in debugging the
root cause. As it affects the production setup every now and then and only
way to resolve this is broker restart. I have also attached jstacks of
activemq at the time the issue was happening.

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Broker-not-delivering-persistent-messages-to-consumer-on-queue-tp4691245p4691325.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Broker not delivering persistent messages to consumer on queue

[xabhi]

Thanks for your prompt response.

As I mentioned before in my observations, I purged the queue and the new
messages that came after purging were also not delivered to the consumer.
And My consumer does not use any selectors or message groups.

Moreover, I believe purging the queue would rule out the case where there
are unconsumed messages due to selectors or message groups. 

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Broker-not-delivering-persistent-messages-to-consumer-on-queue-tp4691245p4691277.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Broker not delivering persistent messages to consumer on queue

[xabhi]

Hi,
I had a weird issue where broker was not delivering persistent messages to
consumer on some queues. I tried following things on one of the affected
queues:

1. I added a new consumer on that queue but it didn't receive any messages.
2. I restarted the original consumer but it didn't receive any messages
after the restart.
3. I purged the messages that were held on the queue but then messages
started accumulating again and broker didn't deliver any of the new
messages. When I purged the expiry count didn't increase neither the dequeue
and dispatch counters.
4. I sent 100 non-persistent messages on that queue, surprisingly it
received those messages.
5. I tried sending 100 persistent messages on that queue, it didn't deliver
anyone of them, all the messages were held by broker.

Only broker restart fixed this issue.

Another weird thing is that this kind of thing (where broker doesn't deliver
messages to queue consumer for persistent messages) happened multiple times
but it got auto resolved after 30 mins or so. But at one point it remained
for about an hour and I had to restart the broker as it is affecting
production setups.

I am attaching thread dumps of activemq process from the time issue was
happening. Can someone please help.

jstack.jstack
http://activemq.2283324.n4.nabble.com/file/n4691245/jstack.jstack  
jstack3.jstack3
http://activemq.2283324.n4.nabble.com/file/n4691245/jstack3.jstack3  

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Broker-not-delivering-persistent-messages-to-consumer-on-queue-tp4691245.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Listener is not able to connect again once maxReconnectAttempts are exhausted by the ActiveMQ layers.

[xabhi]

I have multiple listeners sharing same connection and the broker url has
maxReconnectAttempts property set to 5.
If I restart the broker, the ActiveMQ layer tries 5 reconnect attempts then
gives up and the listener throws up connection refused exception and then
tries to refresh the jms connection and says its refreshed but there is not
connection made to ActiveMQ and the listener again throws up connection
refused exception and again refreshes the jms connection but fails. This
process continues indefinitely.
However, I use single listener and no sharing of connection, this works
fine. The ActiveMQ layers tries 5 reconnect attempts fails, listener throws
up exception and again triggers reconnect attempts which does not happens in
case of multiple listener.
This issue doesn't happen with ActiveMQ 5.8.0

I am attaching some error logs that I observed. errorlogs.txt
http://activemq.2283324.n4.nabble.com/file/n4690929/errorlogs.txt  

Steps to reproduce this issue:

bean id=connectionFactory
class=org.apache.activemq.ActiveMQConnectionFactory
property name=quot;brokerURLquot;
value=quot;failover:(tcp://lt;hostname1:61616?wireFormat.maxInactivityDuration=12,tcp://hostname2:61616?wireFormat.maxInactivityDuration=12)?randomize=falsemaxReconnectAttempts=5
/
/bean
bean id=jmsConnectionFactory
class=org.apache.activemq.pool.PooledConnectionFactory
destroy-method=stop
property name=connectionFactory ref=connectionFactory /
/bean
amq:topic id=testTopic physicalName=test.topic/

bean id=jmsContainer1
class=org.springframework.jms.listener.DefaultMessageListenerContainer
p:connectionFactory-ref=jmsConnectionFactory
p:destination-ref=testTopic
p:pubSubDomain=true
p:subscriptionDurable=false
p:sessionTransacted=true
p:messageListener-ref=consumer /
bean id=jmsContainer2
class=org.springframework.jms.listener.DefaultMessageListenerContainer
p:connectionFactory-ref=jmsConnectionFactory
p:destination-ref=testTopic
p:pubSubDomain=true
p:subscriptionDurable=false
p:sessionTransacted=true
p:messageListener-ref=consumer /
Use this configuration and start the application.
Now restart the ActiveMQ broker once the listeners successfully connect.



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Listener-is-not-able-to-connect-again-once-maxReconnectAttempts-are-exhausted-by-the-ActiveMQ-layers-tp4690929.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Consumer not able to consumer messages from queue

[xabhi]

Hi,
A similar issue happened again in my ActiveMQ broker setup where the broker
didn't deliver any messages to a queue consumer for about an hour. I could
see some of the msgs being held on the broker and the count kept on
increasing during this time though other destinations were working fine and
messages were getting enqueued and dequeued. The message TTL was set for the
messages being set to 2 hrs on this queue.

I am not able explain this behavior in my setup. When does the broker holds
messages and what else should I be looking for? After 1 hour or so all the
messages were delivered and consumer again started receiving messages.

Please help!

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Consumer-not-able-to-consume-messages-from-queue-tp4689594p4690146.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Consumer not able to consumer messages from queue

[xabhi]

Hi Art,
Can you also please take a look at 
http://activemq.2283324.n4.nabble.com/Consumer-not-able-to-consume-messages-from-queue-tp4689594p4689835.html
http://activemq.2283324.n4.nabble.com/Consumer-not-able-to-consume-messages-from-queue-tp4689594p4689835.html
  

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Consumer-not-able-to-consume-messages-from-queue-tp4689594p4689943.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Consumer not able to consumer messages from queue

[xabhi]

The producer is not blocked as the msgHeld value keeps increasing, its the
consumer that stops consuming messages.

Also, I was under the impression that it wasn't possible to set message
TTL 
on the broker and that it could only be set by the producer;

Yes, sorry about that it can be set by producer only, but you can do that by
overriding send method of BrokerPluginSupport class to set message TTL when
broker receives a message (which we do in cases where producer hasn't
specified any message TTL value).

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Consumer-not-able-to-consume-messages-from-queue-tp4689594p4689858.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Consumer not able to consumer messages from queue

[xabhi]

Thanks for the reply.

I have disabled producer flow control on both topics and queues in my broker
configuration and I have message TTL specified on broker side.

The destination on which heartbeats are sent is a Queue on which 5
concurrent consumers are listening and the reply to destination is a topic
on which there are multiple subscribers.

When all this went down the topic destination enqueue count was not
increasing. 

I am pasting the code for consumer and producer:

*Producer:*

public boolean publish()
{
String message = Heartbeat message;
boolean responseReceived = false;

Connection connection = null;
Session session = null;

try
{
  connection = myJmsTemplate.getConnectionFactory().createConnection();
  session= connection.createSession(transacted, ackMode);

  String correlationId = null;
  Long   timeStamp = System.currentTimeMillis();
  Random random= new Random(timeStamp);

  Integer randomPart = random.nextInt(Integer.MAX_VALUE);
  LongthreadId   = Thread.currentThread().getId();
  correlationId  = threadId + _ + timeStamp + _ + randomPart;

  String messageSelector = JMSCorrelationID=' + correlationId + ';
  MessageConsumer responseConsumer =
session.createConsumer(receiveDestination, messageSelector);
  connection.start();

  // send a text message to broker
  myJmsTemplate.send(sendDestination, new
SimpleTextMessageCreator(message, receiveDestination, correlationId));

  LOG.debug(Waiting for message with  + messageSelector +  for  +
DEFAULT_TIMEOUT +  ms);

  // check for response from broker, DEFAULT_TIMEOUT is 60s.
 TextMessage responseMessage = (TextMessage)
responseConsumer.receive(DEFAULT_TIMEOUT);
 if (responseMessage != null)
 {
 if (!responseMessage.getJMSCorrelationID().equals(correlationId)) {
 String errorMsg =
 Invalid correlation id in response message!!!  +
 Expected :  + correlationId +
  but received :  + responseMessage.getJMSCorrelationID();

 LOG.error(errorMsg);
 responseReceived = false;
 }
 else {
 LOG.debug(Recieved the response back:  +
responseMessage.getText());
 LOG.debug(Correlation id of response message :  +
responseMessage.getJMSCorrelationID());
 responseReceived = true;
 }
 }
}
catch (JMSException e)
{
 LOG.error(Error interacting with broker, e);
}
catch (Throwable t) {
 LOG.warn(Publish encountered unknown exception., t);
}
finally {
 JmsUtil.closeConnection(connection, session,
this.getClass().getName());
}
return responseReceived;
}


*Listener/Consumer:*

public class HeartBeatListener implements SessionAwareMessageListener
{

 private final Log LOG = LogFactory.getLog(this.getClass());

 @Override
 public void onMessage(Message message, Session session) throws
JMSException
 {
 if (!(message instanceof TextMessage)) {
 throw new IllegalArgumentException(Message must be of type
TextMessage:  + message);
 }

 String replyTextMessage = Heartbeat Ack.;

 try
 {
 TextMessage textMessage = (TextMessage) message;
 String msg = textMessage.getText();

 LOG.debug(Received heart beat message :  + msg);

 // Send the response to the destination specified by the
 // 'JMSReplyTo' field of the received message.
 Destination responseDest = message.getJMSReplyTo();
 if (responseDest != null)
 {
 LOG.debug(Sending response to destination +
responseDest.toString());

 // Setup a message producer for the above destination
 MessageProducer producer =
session.createProducer(responseDest);
 TextMessage responseMessage =
session.createTextMessage(replyTextMessage);


responseMessage.setJMSCorrelationID(message.getJMSCorrelationID());

 // Send the response back
 producer.send(responseMessage);
 LOG.debug(Heart Beat Response Sent:  + responseMessage);
 }
 }
 catch (JMSException e)
 {
 LOG.error(Error while processing the message  + message, e);
 }
 }
}
This listener is used in a DMLC.

Hi art,
Here the session is created on the producer side (in the publish() function)
and on listener side a session aware listener is used. My question was that
what if the session on publisher side is closed before the session aware
listener is even called or is in process of executing onMessage()?

Please tell me if I am doing anything wrong here?

Thanks for all the help.
-Abhi




--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Consumer-not-able-to-consume-messages-from-queue-tp4689594p4689835.html

Should I still use idleTimeout=0 which ActiveMQ 5.10.0 after the bug fix in ActiveMQ 5.8.0

[xabhi]

There was this bug in ActiveMQ v 5.8.0 (
https://issues.apache.org/jira/browse/AMQ-4366
https://issues.apache.org/jira/browse/AMQ-4366  ) which was worked around
by setting idleTimeout to 0 in connection factory. I have upgraded my setup
to ActiveMQ 5.10.0.

I am wondering whether to keep this setting or remove it to go back to
defaults. What could be the side-effects of setting it to 0? What will be
the benefits of reverting back to default 30s?

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Should-I-still-use-idleTimeout-0-which-ActiveMQ-5-10-0-after-the-bug-fix-in-ActiveMQ-5-8-0-tp4689846.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Consumer not able to consumer messages from queue

[xabhi]

Please help i am not able to get anywhere with this issue.

Atleast tell me what more information to collect and what things to look out
for next time this issue happens which can help to resolve/debug this issue.

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Consumer-not-able-to-consume-messages-from-queue-tp4689594p4689783.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Consumer not able to consumer messages from queue

[xabhi]

Thanks for the reply. I don't have the stack trace currently with me and I
have not been able to reproduce this issue.
In my broker I log hourly destination stats and for this particular topic I
observed that the messages consumption doesn't stop immediately rather it
happens overtime.

[destName: queue://app.heartBeat | enqueueCount: 160014 | dequeueCount:
160014 | dispatchCount: 160014 | expiredCount: 0 | inflightCount: 0 |
msgsHeld: 0 | msgsCached: 0 | memoryPercentUsage: 0 | memoryUsage: 0 |
memoryLimit: 134217728 | avgEnqueueTimeMs: 23540.421475620882 |
maxEnqueueTimeMs: 3159190 | minEnqueueTimeMs: 1 | currentConsumers: 10 |
currentProducers: 0 | blockedSendsCount: 0 | blockedSendsTimeMs: 0 |
minMsgSize: 1033 | maxMsgSize: 1074 | avgMsgSize: 1057.1182958991087 |
totalMsgSize: 169153727] – all messages consumed till now.

[destName: queue://app.heartBeat | enqueueCount: 160408 | dequeueCount:
160274 | dispatchCount: 160274 | expiredCount: 0 | inflightCount: 0 |
msgsHeld: 134 | msgsCached: 0 | memoryPercentUsage: 0 | memoryUsage: 0 |
memoryLimit: 134217728 | avgEnqueueTimeMs: 23502.23844790796 |
maxEnqueueTimeMs: 3159190 | minEnqueueTimeMs: 1 | currentConsumers: 10 |
currentProducers: 0 | blockedSendsCount: 0 | blockedSendsTimeMs: 0 |
minMsgSize: 1033 | maxMsgSize: 1074 | avgMsgSize: 1057.1288090369558 |
totalMsgSize: 169571918] – messages consumed 260/394, held - 134

[destName: queue://app.heartBeat | enqueueCount: 160697 | dequeueCount:
160462 | dispatchCount: 160462 | expiredCount: 0 | inflightCount: 0 |
msgsHeld: 235 | msgsCached: 0 | memoryPercentUsage: 0 | memoryUsage: 0 |
memoryLimit: 134217728 | avgEnqueueTimeMs: 24854.09577345415 |
maxEnqueueTimeMs: 3159190 | minEnqueueTimeMs: 1 | currentConsumers: 10 |
currentProducers: 0 | blockedSendsCount: 0 | blockedSendsTimeMs: 0 |
minMsgSize: 1033 | maxMsgSize: 1074 | avgMsgSize: 1057.1415956738458 |
totalMsgSize: 169879483] – messages consumed 188/289

[destName: queue://app.heartBeat | enqueueCount: 160979 | dequeueCount:
160462 | dispatchCount: 160462 | expiredCount: 0 | inflightCount: 0 |
msgsHeld: 517 | msgsCached: 0 | memoryPercentUsage: 0 | memoryUsage: 0 |
memoryLimit: 134217728 | avgEnqueueTimeMs: 24854.09577345415 |
maxEnqueueTimeMs: 3159190 | minEnqueueTimeMs: 1 | currentConsumers: 10 |
currentProducers: 0 | blockedSendsCount: 0 | blockedSendsTimeMs: 0 |
minMsgSize: 1033 | maxMsgSize: 1074 | avgMsgSize: 1057.1541691773461 |
totalMsgSize: 170179621] – messages consumed 0/282

After this no messages are consumed. Another thing is that the usual hourly
heartbeat message rate is 492 messages/hour but it gradually decreases
during this time to 282.

In my producer, I create a consumer and wait for the acknowledgement using
consumer.receive() call with 60s timeout and at listener side I send back
the acknowledgement for heartbeat by creating a producer using the same
underlying session that I get in session aware listener. I am using
selectors for this kind heartbeat messaging and acknowledgement (with
persistent messages).

I have one question, The producer closes the session on its side under which
it sent the heartbeat message once it fails to receive an acknowledgement
(may be due to timeout), what will be the behavior of session aware listener
on consumer side (which will use same session to send back the ack)?

Thanks,
Abhi


Does this behavior points in some direction I should look for?



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Consumer-not-able-to-consumer-messages-from-queue-tp4689594p4689632.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Consumer not able to consumer messages from queue

[xabhi]

Hi,
I have a session aware message listener inside a DMLC with 5 concurrent
consumers consuming messages from a queue. This listener will send a message
everytime it receives a messages using the same session object to another
topic by creating a producer.

This setup runs fine for around 15 days and then suddenly the consumers stop
consuming messages from the queue and this continues until the broker is
restarted.

I think my issue may be related to this
http://activemq.2283324.n4.nabble.com/ActiveMQ-message-dequeuing-hangs-td4681366.html
but I am not sure.

Where should I start looking for the possible issues?

I am using spring v4.0.5 and ActiveMQ 5.10 and activemq's pooled connection
factory

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Consumer-not-able-to-consumer-messages-from-queue-tp4689594.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: LevelDB Documentation

[xabhi]

Yeah, there is no clear documentation about levelDB in ActiveMQ. Please help
us to know what is the mechanism and how indexes are stored in levelDB.


Thanks,
Anuj



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/LevelDB-Documentation-tp4679547p4679574.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: LevelDB: Multiple Persistent Adapters

[xabhi]

Hi,

I am also thinking about moving to levelDB but its not clear how to use
multiple persistent adapters for each destination just like KahaDB. 

Can somebody answer how to avail this requirement in LevelDB or if it is
available or not ?

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/LevelDB-Multiple-Persistent-Adapters-tp4679490p4679581.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

LevelDB Documentation

[xabhi]

Hi,

I am trying to use levelDB in ActiveMQ5.9 but did not find any documentation
of levelDB for ActiveMQ. 
I have gone through http://activemq.apache.org/leveldb-store.html already
but this page just give us very abstract details about levelDB.

I want to know detail like 
how key-value pair is used in levelDB 
Fast index snapshots using hard links
Concurrent reads

Can someone please recommend me any helpful document for ActiveMQ using
KahaDB.

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/LevelDB-Documentation-tp4679547.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Slow KahaDB access

[xabhi]

Please respond

-
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Slow-KahaDB-access-tp4677915p4678345.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Slow KahaDB access

[xabhi]

Can anybody answer the queries posted in previous post.

-
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Slow-KahaDB-access-tp4677915p4678236.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Slow KahaDB access

[xabhi]

Hi,
I have few more queries:

1. IIUC, If there is only one producer sending persistent messages, then
journal writes will happen for every message and Slow KahaDB access logging
will be done for every message.

2. In what cases does Slow KahaDB access: Journal read took  will logged?
In which cases a journal read operation takes place?

3. Does LevelDB also have a similar kind of Slow file system access logging?
If not then why it is dropped?

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Slow-KahaDB-access-tp4677915p4678185.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Slow KahaDB access

[xabhi]

Hi,
I have read that page and I could only find that the logging will be done
after cleanup or checkpoint interval.
But its not clear what happens when a persistent message arrives at broker.
Does journal write happens as soon as a persistent message arrives(i.e.
separately for every persistent message) or a bunch of messages are
persisted together?


Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Slow-KahaDB-access-tp4677915p4678030.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Slow KahaDB access

[xabhi]

Can somebody from activemq dev answer these queries please?

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Slow-KahaDB-access-tp4677915p4677981.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Slow KahaDB access

[xabhi]

Hi,
How frequently are these slow kahadb messages are logged?

I checked source code and found that these are logged during cleanup,
journal append and journal read.
I know cleanup happens only after cleanup interval but I know nothing about
how frequently journal append and journal read happens. 
Does journal append and read happens for every new persistent message that
is received by the broker? or it happens after some time interval, if yes
can it be configured? 

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Slow-KahaDB-access-tp4677915.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Kahadb diagnostic logging

[xabhi]

Hi christian,
How frequently this diagnostic logging happens? Does it happen for every
persistent message received by broker? Could you please tell me for what
operations i can expect these kind of log messages?

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Kahadb-diagnostic-logging-tp4677536p469.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Kahadb diagnostic logging

[xabhi]

Hi,
How much burden does kahadb diagnostic logging puts on message broker?
I want to catch filesystem issues as soon as possible and I found this
diagnostic logging option.

Has anybody used it? Will it cause any additional burden on broker?

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Kahadb-diagnostic-logging-tp4677536.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Message Dequeue count in jconsole 0 even after messages are recieved and consumed by subscribers

[xabhi]

https://issues.apache.org/jira/browse/AMQ-4965
https://issues.apache.org/jira/browse/AMQ-4965  

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Message-Dequeue-count-in-jconsole-0-even-after-messages-are-recieved-and-consumed-by-subscribers-tp4675875p4676196.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Message Dequeue count in jconsole 0 even after messages are recieved and consumed by subscribers

[xabhi]

Thanks for reply christian. 
Yeah it would be good if the metrics work in same way for both durable and
non-durable subscribers. It would also be nice if it was documented
somewhere in ActiveMQ docs as I couldn't find any information about this
behavior in ActiveMQ docs.




--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Message-Dequeue-count-in-jconsole-0-even-after-messages-are-recieved-and-consumed-by-subscribers-tp4675875p4676143.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Message Dequeue count in jconsole 0 even after messages are recieved and consumed by subscribers

[xabhi]

Can anybody from ActiveMQ team explain this behavior?




--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Message-Dequeue-count-in-jconsole-0-even-after-messages-are-recieved-and-consumed-by-subscribers-tp4675875p4676074.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Message Dequeue count in jconsole 0 even after messages are recieved and consumed by subscribers

[xabhi]

Thanks for reply artnassef.
My question is  Why dequeue count for a topic is not zero in cases where I
have non-durable subscribers whereas it is zero if my topic has only durable
subscriber. 

Can anyone explain this behavior?




--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Message-Dequeue-count-in-jconsole-0-even-after-messages-are-recieved-and-consumed-by-subscribers-tp4675875p4676032.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Hung ActiveMQ broker and processes are blocking

[xabhi]

I looked at stack trace and it has something to do with inactivity monitor.
Some one from ActiveMQ development team can help us here.



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Hung-ActiveMQ-broker-and-processes-are-blocking-tp4675874p4675967.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Message Dequeue count in jconsole 0 even after messages are recieved and consumed by subscribers

[xabhi]

Thanks for the reply.

I am looking at the topic mbean. The dequeue count in topic mbean is 0 even
if the subscribers is receiving and processing messages. Does this mean that
messages are not removed from the topic and are still available?

How come dequeue counter is not zero in cases where I have non-durable
subscribers? What is the difference here ? I am not able to understand this.

Thanks.
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Message-Dequeue-count-in-jconsole-0-even-after-messages-are-recieved-and-consumed-by-subscribers-tp4675875p4675906.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Message Dequeue count in jconsole 0 even after messages are recieved and consumed by subscribers

[xabhi]

I found this  link http://fusesource.com/forums/thread.jspa?messageID=16510  
and performed a little test and found that 

1. The dequeue count increases for non-durable subscribers (session
transacted was true).
2. The dequeue count does not increase for durable subscribers (session
transacted was true) and other counts like enqueue, dispatch increases
normally. The strange part is inflight count is zero and dequeue count is
also zero but dispatch (= inflight+dequeue) is equal to enqueue. How is this
possible?

Is this an expected behavior ? can anybody from activemq dev team confirm
this behavior??

There is no documentation regarding this. Ideally, i would expect the
dequeue count to increase in case of durable subscribers also. 

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Message-Dequeue-count-in-jconsole-0-even-after-messages-are-recieved-and-consumed-by-subscribers-tp4675875p4675885.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Message Dequeue count in jconsole 0 even after messages are recieved and consumed by subscribers

[xabhi]

Hi,
I have producer which constantly is sending messages to the topic and i can
see that messages are enqueued in the topic and dispatched also and i see
that there are subscribers which are receiving messages. The problem is that
the dequeue count in jconsole is still shown as zero even though the
subscribers are receiving messages.

Has anybody faced a similar problem before? Please help me, I am really
stuck on this.

Thanks,
Abhi



--
View this message in context: 
http://activemq.2283324.n4.nabble.com/Message-Dequeue-count-in-jconsole-0-even-after-messages-are-recieved-and-consumed-by-subscribers-tp4675875.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.