Re: [zeromq-dev] Releasing RAM in Chunks

2012-07-03 Thread Paul Colomiets
Hi Anatoly, On Mon, Jul 2, 2012 at 9:48 PM, Anatoly tolit...@gmail.com wrote: ØMQ Crowd, Reading messages from a socket, placing them on the zmq (PUSH). On the other side reading messages of off the queue (PULL) and persisting them in to DB. If we get millions of messages, ØMQ

[zeromq-dev] Releasing RAM in Chunks

2012-07-02 Thread Anatoly
ØMQ Crowd, Reading messages from a socket, placing them on the zmq (PUSH). On the other side reading messages of off the queue (PULL) and persisting them in to DB. If we get millions of messages, ØMQ takes X GB of RAM (since the pushing in this case is at much higher speed than

Re: [zeromq-dev] Releasing RAM in Chunks

2012-07-02 Thread Chuck Remes
On Jul 2, 2012, at 1:48 PM, Anatoly wrote: ØMQ Crowd, Reading messages from a socket, placing them on the zmq (PUSH). On the other side reading messages of off the queue (PULL) and persisting them in to DB. If we get millions of messages, ØMQ takes X GB of RAM (since the

Re: [zeromq-dev] Releasing RAM in Chunks

2012-07-02 Thread Pieter Hintjens
Anatoly, The standard causes for memory exhaustion are, in order: * You're forgetting to close messages, or doing it at the wrong time. * Your consumer is unable to process messages at full speed, so the queue builds up. In the second case you need to decide how you want to handle the excess.