Keith Wall created QPID-7918:
--------------------------------
Summary: [Java Broker] Respond to low heap memory condition by
throttling producers.
Key: QPID-7918
URL: https://issues.apache.org/jira/browse/QPID-7918
Project: Qpid
Issue Type: Improvement
Components: Java Broker
Reporter: Keith Wall
Currently, if the cumulative size of messages headers/contents cached in direct
memory exceeds a threshold, the Java Broker responds by flowing the messages to
disk and evicting them from memory. This guards the condition of excessive
*message payload* causing an out-of-memory situation.
However the Broker also maintains a linked list representing every queues in
the JVMs heap memory. Each message is represented by a small node in the
linked list. The Broker's design currently means that the entire linked list
must reside in (heap) memory (even if the message's content has been paged out
by the flow to disk mechanism). This gives rise to the the possibility that
the linked lists backing the queue could exhaust all heap and the Broker could
fail with an OOM (heap) condition.
The long term aim is to allow sections linked list to be paged out to eliminate
this problem, but this would necessitate a deep redesign.
As a interim measure, it appears that the the JVM's Memory bean
(MemoryPoolMXBean) could be used to notify the Broker when after a garbage
collection cycle heap memory usage is above a threshold. The Broker could
respond to this event by throttling producer, thus allow consumes to catch up
and hopefully avoiding the failure. The Broker would need to relinquish flow
control once memory usage had fallen again.
Consideration would need to be given for different JVMs and different GC
implementations. Consideration must also be given for use cases where the
Broker is embedded.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
