I think a lot of the extra overhead is coming from some overly generous default allocation sizes that ended up getting used in a lot of places. I've adjusted these down. If you can retry your testing with trunk I'm hoping you should see some improvement.
--Rafael On Fri, May 2, 2014 at 6:29 AM, Rafael Schloming <[email protected]> wrote: > On Thu, May 1, 2014 at 8:14 PM, Michael Goulish <[email protected]>wrote: > >> >> >> >> >> ----- Original Message ----- >> > On 05/01/2014 08:55 PM, Rafael Schloming wrote: >> > > On Thu, May 1, 2014 at 3:42 PM, Michael Goulish <[email protected]> >> > > wrote: >> > >> I tried firing up my messenger-based receivers, each subscribing to >> 100 >> > >> addresses, then 200, 300, 400, 500. The results are consistent >> across >> > >> that range, and show that each extra address costs 115 KB. ( Looking >> > >> only at resident-set size. ) >> > >> >> > >> So when I tried to do a total of 1,000,000 addrs on one box, I did >> > >> indeed overwhelm my memory. That would come to 115 GB, which >> > >> would have been more than double my physical mem. >> > >> >> > >> Please note I did not actually send any messages. A router was >> running >> > >> for these receivers to attach to, but no senders were running. >> > >> >> > >> Does 115 KB per subscribed addr seem fairly reasonable? >> > >> >> > > >> > > No, that seems quite excessive. Can you trace where the memory is >> actually >> > > coming from? >> > >> > Just for comparison, a qpid::messaging process with 1000 subscriptions >> > over AMQP 1.0 uses 48MB on my laptop. A similar process using AMQP 0-10 >> > uses 8MB. >> > >> > >> >> Yikes. A 40 KB per subscription added cost. >> That's because 1.0 is ten times as good as 0.10 . >> >> Just off the top of your head -- is there anything inherent >> in 1.0 that would make you expect that kind of difference? >> > > Definitely not, the overhead should be comparable. > > --Rafael >
