It's not something I've ever heard described, so it might be a known limitation but if so it's not a very well-publicized one. Can you please create an enhancement request in JIRA for enabling batching in AMQP?
Thanks for creating the reproducer, by the way. That greatly speeds up the time from issue submission to resolution. Tim On Mon, Sep 2, 2019, 7:07 PM Brad Harvey <brad.har...@gbst.com> wrote: > Hi, > > I have observed that using batches does not improve the performance of > ActiveMQ 5's AMQP connector. I have found this for consumers and producers > using JMS transacted sessions, and also for consumers with > client_acknowledge every N messages. The performance does not change at > all compared to individual messages. I suspect that every message is still > acked individually. > > Is this a known limitation of the amqp connector with activeMQ 5? > > This problem does not occur with Active MQ 5 Openwire, nor does it occur > with Artemis AMQP. After initially observing the problem with Camel > sjms-batch + QPID JMS, I was able to use quiver to reproduce with QPID JMS > and QPID messaging CPP. > > I have not tested whether the transactional behaviour is correct (i.e. > atomic). > > Quiver is available here: > https://github.com/ssorj/quiver > > I used the quiver docker image. I was testing with 6600 byte durable > messages, "out of the box" local PC activemq 5.15.9 / artemis 2.9.0 > installs. The problem is also reproduceable on AmazonMQ. > > Following are some example quiver commands and some test results from a > laptop - intended to show relative throughput for no transactions vs > transactions rather than any absolute performance numbers. > > Baseline test cases - Openwire & AMQP have similar performance for largish > durable messages when sent individually. > > <raw> > > # activemq-jms, activemq openwire or artemis openwire, transaction size > 1000 > quiver --verbose --arrow activemq-jms --body 6600 --count 10000 --durable > tcp://hostname:61616/q0 > > activemq results: > > Sender ......................................... activemq-jms > Receiver ....................................... activemq-jms > Address URL ...................... tcp://hostname:61616/q0 > Output files ........................... /tmp/quiver-i7gry34k > Count ................................................ 10,000 messages > Body size ............................................. 6,600 bytes > Credit window ......................................... 1,000 messages > Flags ............................................... durable > > RESULTS > > Count ................................................ 10,000 messages > Duration ............................................... 22.8 seconds > Sender rate ............................................. 438 messages/s > Receiver rate ........................................... 438 messages/s > End-to-end rate ......................................... 438 messages/s > > > artemis results: > > CONFIGURATION > > Sender ......................................... activemq-jms > Receiver ....................................... activemq-jms > Address URL ...................... tcp://hostname:61616/q0 > Output files ........................... /tmp/quiver-eco6j388 > Count ................................................ 10,000 messages > Body size ............................................. 6,600 bytes > Credit window ......................................... 1,000 messages > Flags ............................................... durable > > RESULTS > > Count ................................................ 10,000 messages > Duration ............................................... 27.1 seconds > Sender rate ............................................. 369 messages/s > Receiver rate ........................................... 369 messages/s > End-to-end rate ......................................... 369 messages/s > > > > # qpid-jms, activemq amqp, transaction size 1000 > quiver --verbose --arrow qpid-jms --body 6600 --count 10000 --durable > amqp://hostname:5672/q0 > > CONFIGURATION > > Sender ............................................. qpid-jms > Receiver ........................................... qpid-jms > Address URL ...................... amqp://hostname:5672/q0 > Output files ........................... /tmp/quiver-fbt12d7g > Count ................................................ 10,000 messages > Body size ............................................. 6,600 bytes > Credit window ......................................... 1,000 messages > Flags ............................................... durable > > RESULTS > > Count ................................................ 10,000 messages > Duration ............................................... 28.3 seconds > Sender rate ............................................. 353 messages/s > Receiver rate ........................................... 354 messages/s > End-to-end rate ......................................... 353 messages/s > > > > # output of remaining individual tests omitted for brevity, results in > same order of magnitude. > > # qpid-messaging-cpp, activemq amqp, transaction size 1000 > quiver --verbose --arrow qpid-messaging-cpp --body 6600 --count 10000 > --durable amqp://hostname:5672/q0 > > # qpid-jms + artemis amqp, transaction size 1000 > quiver --verbose --arrow qpid-jms --body 6600 --count 10000 --durable > amqp://hostname:61616/q0 > > # qpid-messaging-cpp, artemis amqp, transaction size 1000 > quiver --verbose --arrow qpid-messaging-cpp --body 6600 --count 10000 > --durable amqp://hostname:61616/q0 > > > </raw> > > These test cases give expected results - transaction size of 1000 > significantly improves performance. > > > <raw> > # activemq-jms, activemq openwire or artemis openwire, transaction size > 1000 > quiver --verbose --arrow activemq-jms --body 6600 --transaction-size 1000 > --count 10000 --durable tcp://hostname:61616/q0 > > activemq: > > CONFIGURATION > > Sender ......................................... activemq-jms > Receiver ....................................... activemq-jms > Address URL ...................... tcp://hostname:61616/q0 > Output files ........................... /tmp/quiver-o893bg1n > Count ................................................ 10,000 messages > Body size ............................................. 6,600 bytes > Credit window ......................................... 1,000 messages > Transaction size ...................................... 1,000 messages > Flags ............................................... durable > > RESULTS > > Count ................................................ 10,000 messages > Duration ................................................ 3.3 seconds > Sender rate ........................................... 3,393 messages/s > Receiver rate ......................................... 3,204 messages/s > End-to-end rate ....................................... 3,007 messages/s > > > artemis: > > CONFIGURATION > > Sender ......................................... activemq-jms > Receiver ....................................... activemq-jms > Address URL ...................... tcp://hostname:61616/q0 > Output files ........................... /tmp/quiver-mp8zwfh0 > Count ................................................ 10,000 messages > Body size ............................................. 6,600 bytes > Credit window ......................................... 1,000 messages > Transaction size ...................................... 1,000 messages > Flags ............................................... durable > > RESULTS > > Count ................................................ 10,000 messages > Duration ................................................ 3.7 seconds > Sender rate ........................................... 3,505 messages/s > Receiver rate ......................................... 3,031 messages/s > End-to-end rate ....................................... 2,702 messages/s > > > > # qpid-jms + artemis amqp, transaction size 1000 > quiver --verbose --arrow qpid-jms --body 6600 --count 10000 > --transaction-size 1000 --durable amqp://hostname:61616/q0 > > CONFIGURATION > > Sender ............................................. qpid-jms > Receiver ........................................... qpid-jms > Address URL ..................... amqp://hostname:61616/q0 > Output files ........................... /tmp/quiver-ok2sjt3s > Count ................................................ 10,000 messages > Body size ............................................. 6,600 bytes > Credit window ......................................... 1,000 messages > Transaction size ...................................... 1,000 messages > Flags ............................................... durable > > RESULTS > > Count ................................................ 10,000 messages > Duration ................................................ 5.7 seconds > Sender rate ........................................... 2,714 messages/s > Receiver rate ......................................... 2,050 messages/s > End-to-end rate ....................................... 1,767 messages/s > > > # qpid-messaging-cpp, artemis amqp, transaction size 1000 > quiver --verbose --arrow qpid-messaging-cpp --body 6600 --count 10000 > --transaction-size 1000 --durable amqp://hostname:61616/q0 > > CONFIGURATION > > Sender ................................... qpid-messaging-cpp > Receiver ................................. qpid-messaging-cpp > Address URL ..................... amqp://hostname:61616/q0 > Output files ........................... /tmp/quiver-8gndofrc > Count ................................................ 10,000 messages > Body size ............................................. 6,600 bytes > Credit window ......................................... 1,000 messages > Transaction size ...................................... 1,000 messages > Flags ............................................... durable > > RESULTS > > Count ................................................ 10,000 messages > Duration ................................................ 3.1 seconds > Sender rate ........................................... 6,039 messages/s > Receiver rate ......................................... 3,351 messages/s > End-to-end rate ....................................... 3,175 messages/s > </raw> > > These two test cases give unexpected results - performance roughly equal > to the "sent individually" tests, but expected performance is similar to > the other transaction size of 1000 tests. > > > <raw> > # qpid-jms, activemq amqp, transaction size 1000 > quiver --verbose --arrow qpid-jms --body 6600 --transaction-size 1000 > --count 10000 --durable amqp://hostname:5672/q0 > > CONFIGURATION > > Sender ............................................. qpid-jms > Receiver ........................................... qpid-jms > Address URL ...................... amqp://hostname:5672/q0 > Output files ........................... /tmp/quiver-0tjnz8sn > Count ................................................ 10,000 messages > Body size ............................................. 6,600 bytes > Credit window ......................................... 1,000 messages > Transaction size ...................................... 1,000 messages > Flags ............................................... durable > > RESULTS > > Count ................................................ 10,000 messages > Duration ............................................... 26.9 seconds > Sender rate ............................................. 455 messages/s > Receiver rate ........................................... 396 messages/s > End-to-end rate ......................................... 372 messages/s > > > > # qpid-messaging-cpp, activemq amqp, transaction size 1000 > quiver --verbose --arrow qpid-messaging-cpp --body 6600 --count 10000 > --transaction-size 1000 --durable amqp://hostname:5672/q0 > > CONFIGURATION > > Sender ................................... qpid-messaging-cpp > Receiver ................................. qpid-messaging-cpp > Address URL ...................... amqp://hostname:5672/q0 > Output files ........................... /tmp/quiver-y4k_139e > Count ................................................ 10,000 messages > Body size ............................................. 6,600 bytes > Credit window ......................................... 1,000 messages > Transaction size ...................................... 1,000 messages > Flags ............................................... durable > > RESULTS > > Count ................................................ 10,000 messages > Duration ............................................... 25.9 seconds > Sender rate ............................................. 459 messages/s > Receiver rate ........................................... 410 messages/s > End-to-end rate ......................................... 387 messages/s > > > </raw> > > > > The information transmitted is intended only for the person or entity to > which it is addressed and may contain confidential and / or privileged > material that may be governed by confidential information provisions > contained in the agreement between GBST and your company. Any disclosure, > copying, distribution, or other use without the express consent of the > sender is prohibited. If you received this in error, please contact the > sender and delete the material from any computer. All rights in the > information transmitted, including copyright, are reserved. Nothing in this > message should be interpreted as a digital signature that can be used to > authenticate a document. No warranty is given by the sender that any > attachments to this email are free from viruses or other defects. >
