[
https://issues.apache.org/jira/browse/QPID-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rudyy resolved QPID-6769.
------------------------------
Resolution: Not A Problem
Assignee: Alex Rudyy
We investigated why number of NonPooledByteBufferRef was growing in the
destruction tests. It is not a memory leak but we found that after message is
flown to disk, the meta data are reloaded immediately into heap on first
invocation of Messsage#getMetaData() (which is on creation of queue entry when
is persistent flag is checked).
It is not a blocker but potentially we can improve the code to avoid
unnecessary overhead with cleaning and reloading of metadata
> [Java Broker] Excessive number of instances of NonPooledByteBufferRef is
> created in destruction tests to exercise memory consumption by persistent
> messages breaching the limits of max heap and direct memory to start flowing
> to disk
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: QPID-6769
> URL: https://issues.apache.org/jira/browse/QPID-6769
> Project: Qpid
> Issue Type: Bug
> Components: Java Broker
> Affects Versions: qpid-java-6.0
> Reporter: Alex Rudyy
> Assignee: Alex Rudyy
> Priority: Blocker
> Fix For: qpid-java-6.0
>
>
> Unexpected number of of instances of NonPooledByteBufferRef is created on
> performing destruction test to exercise memory usage on publishing and
> consuming messages with already enqueued large number of persistent messages
> exceeding its memory capacity in patterns that cause both some messages to
> pass almost immediately though the broker, and others to be retained over a
> larger period of time before consumption.
> # I started broker with default memory settings (512M of Heap and 1.5G of
> direct memory), published 290000 of 20k persistent messages using
> StressTestClient and verified that memory heap usage reached the limit and
> messages started to flow to disk:
> {noformat}
> java -Xmx500m -cp "..." -Dqpid.dest_syntax=BURL -Dqpid.amqp.version=0-9
> org.apache.qpid.tools.StressTestClient messagecount=290000 size=20000
> persistent=true consumers=0 random=false reportmod=1000
> {noformat}
> # Broker heap dump was collected after full GC.
> # Two other StressTestClient processes have been started: one publishing into
> queue with a lot of messages and another publishing into empty queue:
> {noformat}
> java -Xmx500m -cp "..." -Dqpid.dest_syntax=BURL -Dqpid.amqp.version=0-9
> org.apache.qpid.tools.StressTestClient messagecount=10 size=20000
> persistent=true random=false repetitions=300 lowprefetch=true
> {noformat}
> {noformat}
> java -Xmx500m -cp "..." -Dqpid.dest_syntax=BURL -Dqpid.amqp.version=0-9
> org.apache.qpid.tools.StressTestClient messagecount=10 size=20000
> persistent=true random=true suffix=_b repetitions=300 lowprefetch=true
> {noformat}
> # Broker heap dump was collected after full GC
> # Repeated 2 steps above 3 more times
> What I noticed in the heap dump that on running steps 3-4 during 3 first
> iterations the number of NonPooledByteBufferRef was growing by ~3000 in each
> iteration. Only on last 4th iterations the number of NonPooledByteBufferRef
> increased on ~1000 (which indicates that GC started to collect old
> NonPooledByteBufferRef).
> The increase in number of NonPooledByteBufferRef on each test iteration looks
> suspicious and requires further investigation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
