On 13 October 2016 at 17:36, rammohan ganapavarapu <rammohanga...@gmail.com>
> Thank you for the link, so no matter how much heap you have you will hit
> the hard limit at some point right?, i thought flow to disk will make
> broker not to crash because of out of memory issue but looks like its not
> the case.
> In my environment we will have dynamic number of producers and consumers so
> its hard to pre measure how much heap we can allocate based on number of
Yeah - currently there is always a hard limit based on the number of "queue
entries". Ultimately there's a trade-off to be had with designing a queue
data structure which is high performing, vs. one which can be offloaded
onto disk. This gets even more complicated for queues which are not strict
FIFO (priority queues, LVQ, etc) or where consumers have selectors.
Ultimately if you are storing millions of messages in your broker then you
are probably doing things wrong - we would expect people to enforce queue
limits and flow control rather than expect the broker to have infinite
capacity (and even off-loading to disk you will still run out of disk space
at some point).
> On Thu, Oct 13, 2016 at 9:05 AM, Lorenz Quack <quack.lor...@gmail.com>
> > Hello Ram,
> > may I refer you to the relevant section of the documentation .
> > As explained there in more detail, the broker keeps a representation of
> > each message in heap even when flowing the message to disk.
> > Therefore the amount of JVM heap memory puts a hard limit on the number
> > message the broker can hold.
> > Kind Regards,
> > Lorenz
> >  https://qpid.apache.org/releases/qpid-java-6.0.4/java-broker
> > /book/Java-Broker-Runtime-Memory.html
> > On 13/10/16 16:40, rammohan ganapavarapu wrote:
> >> Hi,
> >> We are doing some load test using java broker 6.0.2 by stopping all
> >> consumers, broker was crashed at 644359 messages. Even if i try to
> >> broker its crashing with the same oom error.
> >> "persistentEnqueuedBytes" : 12731167222,
> >> "persistentEnqueuedMessages" : 644359,
> >> "queueDepthBytes" : 12731167222,
> >> "queueDepthMessages" : 644359,
> >> "totalDequeuedBytes" : 0,
> >> "totalDequeuedMessages" : 0,
> >> "totalEnqueuedBytes" : 12731167222,
> >> "totalEnqueuedMessages" : 644359,
> >> JVM settings of broker: -Xmx512m -XX:MaxDirectMemorySize=1536m
> >> "broker.flowToDiskThreshold" : "644245094",
> >> So theoretically broker should flow those messages to disk after the
> >> threshold right then broker shouldn't have caused OOM exception right?
> >> i
> >> have to do any other tuning?
> >> Thanks,
> >> Ram
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
> > For additional commands, e-mail: users-h...@qpid.apache.org