In an atempt to improve the throughput of my system I did various tests. I
started with a simple setup:
- 1 Topic, non persistent
- 1 Producer
- 1 Consumer
- Messge size: 4k
- Active MQ 5.12.1
My system is reasonably fast (8 cores, 32GB memory, SSD) - still I only
manage to send & receive about 45.000 messages per second.
Basically my producer looks like this:
...
ConnectionFactory factory = new
ActiveMQConnectionFactory("tcp://loclhost:61616");
Connection connection = factory.createConnection();
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
producer = session.createProducer(null);
connection.start();
for (long i = 0; i < 1000000; i++) {
final Topic topic = session.createTopic(topicName);
producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
BytesMessage message = session.createBytesMessage();
message.writeBytes(someByteArray); // size 4k
producer.send(topic, message);
}
...
I measured the time that the consumer spends in its onMessage() method and
it's only about 1% of its total runtime (so consuming the messages is not a
bottleneck (no surprise, because of BytesMessages)).
45.000 messages/second does not sound *that* much to me - is this the
maximum I can expect to get from Active MQ?
I did some further tests increasing the number of topic consumers - and
noticed that my msg/sec drop. But the total amount of mes/sec that pass
through my system stays roughly the same (OK, with some shrinkage/friction):
producer: 45.000
consumer: 45.000
--------- -> 90.000 total
producer: 28.700
consumer: 28.700
consumer: 28.700
--------- -> 86.100 total
producer: 20.400
consumer: 20.400
consumer: 20.400
consumer: 20.400
--------- -> 81.600 total
producer: 15.700
consumer: 15.700
consumer: 15.700
consumer: 15.700
consumer: 15.700
--------- -> 78.500 total
What I had expected would look more like this:
producer: 43.000
consumer: 43.000
consumer: 43.000
consumer: 43.000
consumer: 43.000
Where does this "global total msg/sec limit" come from? Where is the
bottleneck?
