> I see there's interest in this feature, so let's figure out how to > implement it. > > Some notes: > > 1. If tcmalloc et al are brittle, do we understand why is it so? Is it > an inherent problem or just a lame implementation?
I think it is just a case for flexibility. You may want to apply different strategies for different programs, or even within the same program. For instance, I remember having an allocator under ACE to create memory from a shared memory pool in some instances. > 2. Douglas' patch stores the pointer to allocator function in a global > variable. That breaks the model of separate contexts (i.e. that several > instances of 0mq can exist in a single process without misinteracting > with each other). We have to think of something more resilient. Perhaps the allocator should be embedded into the context. You could even create separate contexts if you wish to have different allocators. > 3. There are at least two use cases here. AFAIU, Douglas want to > custom-allocate the message bodies while Chunk want to custom-allocate > everything. We should think a bit more about whether one use case > subsumes the other or whether they are two separate use cases with two > separate solutions. Maybe you could provide two args to the allocator when allocating memory: the size in bytes and a hint. This hint could be used by the allocator to decide upon it: use different memory pools for different parts of the program, etc. Then, you could use the allocator for everything in ZeroMQ itself and in programs using ZeroMQ, assuming ZeroMQ will have a default implementation of the allocator. If we go this route, it could also be the chance to use a couple of macros for interfacing with the allocator: ZMQ_ALLOC(size, hint); ZMQ_REALLOC(old_size, new_size, hint); ZMQ_FREE(size, hint); The point is that the macros can (maybe only in debugging mode) decorate the call by passing the file name and line number to the allocator. I have found this invaluable when debugging memory problems. Unluckily, it has been too long since the last time I did something like this; I would have to force myself to get back up to speed in order to contribute with more than simple ideas. Finally, and so it does not get lost in the noise: I think that a similar idea should be applied to the strategy for swapping messages to disk when the HWM is reached. -- Gonzalo Diethelm _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
