[
https://issues.apache.org/jira/browse/QPID-7753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Keith Wall resolved QPID-7753.
------------------------------
Resolution: Fixed
> Sparsely occupied message buffers may lead to java.lang.OutOfMemoryError:
> Direct buffer memory
> ----------------------------------------------------------------------------------------------
>
> Key: QPID-7753
> URL: https://issues.apache.org/jira/browse/QPID-7753
> Project: Qpid
> Issue Type: Bug
> Components: Java Broker
> Affects Versions: qpid-java-6.0, qpid-java-6.1
> Reporter: Keith Wall
> Assignee: Keith Wall
> Fix For: qpid-java-broker-7.0.0
>
> Attachments: flow-to-disk-based-on-used-direct-memory-6-0-x.diff
>
>
> Some Broker usage patterns can lead to the Broker failing with a
> "java.lang.OutOfMemoryError: Direct buffer memory" error.
> For the condition to manifest a producing application needs to use a single
> connection for the production of messages. Some messages need to be consumed
> quickly whilst others remain on the Broker. This pattern might result from:
> # the consuming application using message selectors to selectively consume
> some messages whilst leaving others on the Broker.
> # the use of 'out of order' queue types (priority/sorted etc) where lower
> priority items are left of the Broker.
> # the producing application routing messages to multiple queues and the
> consuming application draining some queues but not others.
> The problem arises owing to the buffering strategy under the IO layer.
> {{NonBlockingConnection}} allocates a {{netInputBuffer}} from pooled direct
> memory of size 256K. This buffer is used for all network reads until the
> space remaining in the buffer is less than the amount required to complete
> the AMQP frame that is currently being read, in which case a new
> netInputBuffer is allocated. The protocol layers identify the message
> payload/message headers as part of AMQP frame parsing and create a
> byte-buffer *views* onto the original input byte buffer. This byte buffer is
> retained by the store until the message is consumed. In the usage pattern
> described above, a single long lived message amongst a stream of shorted
> lived messages causes an entire 256K buffer chunk to be retained. Qpid
> cannot dispose or reuse the buffer until it is entirely unoccupied.
> The flow to disk feature is designed to prevent excessive direct memory use
> by flushing messages to disk if thresholds are breached. Flow to disk does
> not help the scenario described above because it considers the total payloads
> of live messages. Its algorithm does not consider direct memory occupancy.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
