On 02/05/2013 14:36, Andrzej Dworak wrote: > Dear colleagues, > > First of all I should have sent that paper more than a year ago, sorry. > Hopefully it will be still of some value... In fact it was only a very > interesting discussion at reddit that reminded me to send the small review we > did. Again, sorry for being late on that. > > http://www.reddit.com/r/programming/comments/1d0pwj/yami4_vs_zeromq/
OK, I'll bite. I happen to work on a graphical product that lives in memory constrained environments, ranging from embedded with tens of MB up to servers with hundreds of GB. What!, Are all these systems memory constrained? Well, yes. Any product that works best when it retains useful stuff in memory can eventually run out of memory, suffer from severe fragmentation and swap space performance. Think of a product like Photoshop doing complex transforms on hundreds of images (I don't work on Photoshop, but everyone's heard of it). As you add more images, a user would hope that life wouldn't become unbearable as swap space gets swamped. And you certainly don't want your application crashing just because you've added one more image to the pile. Memory management is therefore a very important design feature of such products. Strategies include using a pooled memory manager (such as http://www.pjsip.org/pjlib/docs/html/group__PJ__POOL__GROUP.htm), memory high water marks, back off and retry allocations, spilling to disk, purging low priority caches, garbage collection, and more. At this point, we want to add ZeroMQ to the mix. Crashes are not allowed, and we'd like our low memory strategies to be successful as much as possible. Given the ZeroMQ policy of asserting on allocation failure, we have to decide on a policy. We could: - do nothing and allow ZeroMQ to allocate out of the heap (rather than our pooled memory manager) and assert when heap gets exhausted. That would work in the higher end systems, but not on embedded. - replace the asserts with an exception. That would at least allow the application to catch the exceptions and fail gracefully. - replace the mallocs with allocations from the pooled memory manager. That would allow us to apply the same low memory handling to these allocations. - allocations for C++ constructors could be replaced by overloading new() in all base classes. - define a new_handler to call back to our memory manager. These things would be enough to make our product work acceptably with ZeroMQ. Some of them are ugly, but all are fairly simple using the C dark arts, and don't require much modification of the ZeroMQ library itself. Ideally, there would be pluggable memory in the library, which would be more work. Eventually, we can hope to provide exactly that, but not for a while. Regards John _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
