Hello, My team is using the Java broker and Java client, version 0.16, and looking to lower the client's memory footprint on our servers. We did some heap analysis and found that the consumption is coming mostly from AMQAnyDestination instances, each having a retained size of 2864B, and since we have 6000 queues on each of our 2 brokers, this amounts to about ~33MB, which is valuable real estate for us. In our analysis we found a few possible optimizations in the Qpid code that would reduce the per-queue heap consumption and which don't seem high risk, and were wondering how feasible these changes are.
The first optimization involves cacheing / resuing redundant maps, while the rest involves making the default value of a variable the constant empty Map or List, instead of a new empty Map or List. 1. In Address / AddressParser, cacheing / reusing the options Maps for queues created with the same options string. (This optimization gives us the most significant savings, but has the highest risk.) All our queues are created with the same options string, which means each corresponding AMQDestination has an Address that has an _options Map that is the same for all queues, i.e., 12K copies of the same map. As far as we can tell, the _options map is effectively immutable, i.e., there is no code path by which an Address’s _options map can be modified. (Is this correct?) So a possible improvement is that in org.apache.qpid.messaging.util.AddressParser, we cache the options map for each options string that we've already encountered, and if the options string passed in has already been seen, we use the stored options map for that Address. This way, for queues having the same options, their Address options will reference the same Map. (For our queues, each Address _options Map takes up 1384 B, so with 12K queues we'd save about 15.5MB.) 2. AMQDestination's _link field -- org.apache.qpid.client.messaging.address.Link$Subscription Subscription's args field is by default a new HashMap with default capacity 16. In our use case it remains empty for all queues. A possible optimization is to set the default value as Collections.emptyMap() instead. As far was we can tell, Subscription.getArgs() is not used to get the map and then modify it. For us this saves 128B per queue. 3. AMQDestination's _targetNode field -- org.apache.qpid.client.messaging.address.Node$ExchangeNode Optimization A: The field _exchangeOptions is never used and can be removed. It's an empty new HashMap with default capacity 16. For us this saves 128B per queue. Optimization B: The parent class Node has a _bindings List that is by default a new ArrayList with the default capacity of 10. In our use case _bindings remains empty for all queues. A possible optimization is to set the default value as Collections.emptyList() instead, and if addBindings() is called, lazily set it as a new empty list first. For us this saves 80B per queue. 4. AMQDestination's _sourceNode field -- org.apache.qpid.client.messaging.address.Node$QueueNode Same 2 optimizations as (3) (with the unused field for QueueNode being _queueOptions). Overall: For us, the 5 optimizations in (2), (3), and (4) would save us 128B x 3 + 80B x 2 = 544B per queue, and (1) would save us 1384B per queue; all together this would save 1928B out of 2864B, or 67%, per queue for us. We'd appreciate feedback on how feasible these changes are and whether we are making any incorrect assumptions. Thanks a lot! Helen
