On 03/24/2011 06:16 PM, Chuck Remes wrote: > True, I am looking for a global replacement for all malloc/free > calls. Even if the existing ones were just replaced by a macro that > was easily modified at compile time that would make me happy.
Yes. If we put aside tcmalloc-style solution, that would be the only remaning solution of the problem. > I don't agree. While the experiments showed that the yqueue memory > was being deallocated as expected, it may have still been causing > some kind of fragmentation that prevented the memory from being > returned back to the OS. > > When I was reading up on jemalloc [1] I noticed a passage early in > the text regarding allocator-induced fragmentation. If the allocator > let's too much fragmentation occur, this "leak" could make the > Facebook application heaps grow beyond the bounds of physical memory > and cause problems. My use-case is very similar in the sense that I > have distributed applications that run for days or weeks at a time > wherein a lot of sockets are created and destroyed over that time > period. There appears to something about the way 0mq is allocating > and freeing memory that creates this fragmentation and slow heap > growth over time. If jemalloc can solve it, then I'm very > interested. > > Note that even though some of the 0mq allocations are 8kB or 12kB, > unless they are page aligned then even these sizes could cause > fragmentation. If even one byte on a page is being held by the > process, it cannot be relinquished to the OS hence the heap > fragmentation. Ok. I have no strong opinion here. We can proceed by allowing to redefine malloc/free at compile time, as discussed above and you can check whether alternate mechanism actually solves your problem. Martin _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
