Hi, I have observed that using batches does not improve the performance of ActiveMQ 5's AMQP connector. I have found this for consumers and producers using JMS transacted sessions, and also for consumers with client_acknowledge every N messages. The performance does not change at all compared to individual messages. I suspect that every message is still acked individually.
Is this a known limitation of the amqp connector with activeMQ 5? This problem does not occur with Active MQ 5 Openwire, nor does it occur with Artemis AMQP. After initially observing the problem with Camel sjms-batch + QPID JMS, I was able to use quiver to reproduce with QPID JMS and QPID messaging CPP. I have not tested whether the transactional behaviour is correct (i.e. atomic). Quiver is available here: https://github.com/ssorj/quiver I used the quiver docker image. I was testing with 6600 byte durable messages, "out of the box" local PC activemq 5.15.9 / artemis 2.9.0 installs. The problem is also reproduceable on AmazonMQ. Following are some example quiver commands and some test results from a laptop - intended to show relative throughput for no transactions vs transactions rather than any absolute performance numbers. Baseline test cases - Openwire & AMQP have similar performance for largish durable messages when sent individually. <raw> # activemq-jms, activemq openwire or artemis openwire, transaction size 1000 quiver --verbose --arrow activemq-jms --body 6600 --count 10000 --durable tcp://hostname:61616/q0 activemq results: Sender ......................................... activemq-jms Receiver ....................................... activemq-jms Address URL ...................... tcp://hostname:61616/q0 Output files ........................... /tmp/quiver-i7gry34k Count ................................................ 10,000 messages Body size ............................................. 6,600 bytes Credit window ......................................... 1,000 messages Flags ............................................... durable RESULTS Count ................................................ 10,000 messages Duration ............................................... 22.8 seconds Sender rate ............................................. 438 messages/s Receiver rate ........................................... 438 messages/s End-to-end rate ......................................... 438 messages/s artemis results: CONFIGURATION Sender ......................................... activemq-jms Receiver ....................................... activemq-jms Address URL ...................... tcp://hostname:61616/q0 Output files ........................... /tmp/quiver-eco6j388 Count ................................................ 10,000 messages Body size ............................................. 6,600 bytes Credit window ......................................... 1,000 messages Flags ............................................... durable RESULTS Count ................................................ 10,000 messages Duration ............................................... 27.1 seconds Sender rate ............................................. 369 messages/s Receiver rate ........................................... 369 messages/s End-to-end rate ......................................... 369 messages/s # qpid-jms, activemq amqp, transaction size 1000 quiver --verbose --arrow qpid-jms --body 6600 --count 10000 --durable amqp://hostname:5672/q0 CONFIGURATION Sender ............................................. qpid-jms Receiver ........................................... qpid-jms Address URL ...................... amqp://hostname:5672/q0 Output files ........................... /tmp/quiver-fbt12d7g Count ................................................ 10,000 messages Body size ............................................. 6,600 bytes Credit window ......................................... 1,000 messages Flags ............................................... durable RESULTS Count ................................................ 10,000 messages Duration ............................................... 28.3 seconds Sender rate ............................................. 353 messages/s Receiver rate ........................................... 354 messages/s End-to-end rate ......................................... 353 messages/s # output of remaining individual tests omitted for brevity, results in same order of magnitude. # qpid-messaging-cpp, activemq amqp, transaction size 1000 quiver --verbose --arrow qpid-messaging-cpp --body 6600 --count 10000 --durable amqp://hostname:5672/q0 # qpid-jms + artemis amqp, transaction size 1000 quiver --verbose --arrow qpid-jms --body 6600 --count 10000 --durable amqp://hostname:61616/q0 # qpid-messaging-cpp, artemis amqp, transaction size 1000 quiver --verbose --arrow qpid-messaging-cpp --body 6600 --count 10000 --durable amqp://hostname:61616/q0 </raw> These test cases give expected results - transaction size of 1000 significantly improves performance. <raw> # activemq-jms, activemq openwire or artemis openwire, transaction size 1000 quiver --verbose --arrow activemq-jms --body 6600 --transaction-size 1000 --count 10000 --durable tcp://hostname:61616/q0 activemq: CONFIGURATION Sender ......................................... activemq-jms Receiver ....................................... activemq-jms Address URL ...................... tcp://hostname:61616/q0 Output files ........................... /tmp/quiver-o893bg1n Count ................................................ 10,000 messages Body size ............................................. 6,600 bytes Credit window ......................................... 1,000 messages Transaction size ...................................... 1,000 messages Flags ............................................... durable RESULTS Count ................................................ 10,000 messages Duration ................................................ 3.3 seconds Sender rate ........................................... 3,393 messages/s Receiver rate ......................................... 3,204 messages/s End-to-end rate ....................................... 3,007 messages/s artemis: CONFIGURATION Sender ......................................... activemq-jms Receiver ....................................... activemq-jms Address URL ...................... tcp://hostname:61616/q0 Output files ........................... /tmp/quiver-mp8zwfh0 Count ................................................ 10,000 messages Body size ............................................. 6,600 bytes Credit window ......................................... 1,000 messages Transaction size ...................................... 1,000 messages Flags ............................................... durable RESULTS Count ................................................ 10,000 messages Duration ................................................ 3.7 seconds Sender rate ........................................... 3,505 messages/s Receiver rate ......................................... 3,031 messages/s End-to-end rate ....................................... 2,702 messages/s # qpid-jms + artemis amqp, transaction size 1000 quiver --verbose --arrow qpid-jms --body 6600 --count 10000 --transaction-size 1000 --durable amqp://hostname:61616/q0 CONFIGURATION Sender ............................................. qpid-jms Receiver ........................................... qpid-jms Address URL ..................... amqp://hostname:61616/q0 Output files ........................... /tmp/quiver-ok2sjt3s Count ................................................ 10,000 messages Body size ............................................. 6,600 bytes Credit window ......................................... 1,000 messages Transaction size ...................................... 1,000 messages Flags ............................................... durable RESULTS Count ................................................ 10,000 messages Duration ................................................ 5.7 seconds Sender rate ........................................... 2,714 messages/s Receiver rate ......................................... 2,050 messages/s End-to-end rate ....................................... 1,767 messages/s # qpid-messaging-cpp, artemis amqp, transaction size 1000 quiver --verbose --arrow qpid-messaging-cpp --body 6600 --count 10000 --transaction-size 1000 --durable amqp://hostname:61616/q0 CONFIGURATION Sender ................................... qpid-messaging-cpp Receiver ................................. qpid-messaging-cpp Address URL ..................... amqp://hostname:61616/q0 Output files ........................... /tmp/quiver-8gndofrc Count ................................................ 10,000 messages Body size ............................................. 6,600 bytes Credit window ......................................... 1,000 messages Transaction size ...................................... 1,000 messages Flags ............................................... durable RESULTS Count ................................................ 10,000 messages Duration ................................................ 3.1 seconds Sender rate ........................................... 6,039 messages/s Receiver rate ......................................... 3,351 messages/s End-to-end rate ....................................... 3,175 messages/s </raw> These two test cases give unexpected results - performance roughly equal to the "sent individually" tests, but expected performance is similar to the other transaction size of 1000 tests. <raw> # qpid-jms, activemq amqp, transaction size 1000 quiver --verbose --arrow qpid-jms --body 6600 --transaction-size 1000 --count 10000 --durable amqp://hostname:5672/q0 CONFIGURATION Sender ............................................. qpid-jms Receiver ........................................... qpid-jms Address URL ...................... amqp://hostname:5672/q0 Output files ........................... /tmp/quiver-0tjnz8sn Count ................................................ 10,000 messages Body size ............................................. 6,600 bytes Credit window ......................................... 1,000 messages Transaction size ...................................... 1,000 messages Flags ............................................... durable RESULTS Count ................................................ 10,000 messages Duration ............................................... 26.9 seconds Sender rate ............................................. 455 messages/s Receiver rate ........................................... 396 messages/s End-to-end rate ......................................... 372 messages/s # qpid-messaging-cpp, activemq amqp, transaction size 1000 quiver --verbose --arrow qpid-messaging-cpp --body 6600 --count 10000 --transaction-size 1000 --durable amqp://hostname:5672/q0 CONFIGURATION Sender ................................... qpid-messaging-cpp Receiver ................................. qpid-messaging-cpp Address URL ...................... amqp://hostname:5672/q0 Output files ........................... /tmp/quiver-y4k_139e Count ................................................ 10,000 messages Body size ............................................. 6,600 bytes Credit window ......................................... 1,000 messages Transaction size ...................................... 1,000 messages Flags ............................................... durable RESULTS Count ................................................ 10,000 messages Duration ............................................... 25.9 seconds Sender rate ............................................. 459 messages/s Receiver rate ........................................... 410 messages/s End-to-end rate ......................................... 387 messages/s </raw> The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and / or privileged material that may be governed by confidential information provisions contained in the agreement between GBST and your company. Any disclosure, copying, distribution, or other use without the express consent of the sender is prohibited. If you received this in error, please contact the sender and delete the material from any computer. All rights in the information transmitted, including copyright, are reserved. Nothing in this message should be interpreted as a digital signature that can be used to authenticate a document. No warranty is given by the sender that any attachments to this email are free from viruses or other defects.
