Alex Rudyy created QPID-6769:
--------------------------------
Summary: [Java Broker] Excessive number of instances of
NonPooledByteBufferRef is created in destruction tests to exercise memory
consumption by persistent messages breaching the limits of max heap and direct
memory to start flowing to disk
Key: QPID-6769
URL: https://issues.apache.org/jira/browse/QPID-6769
Project: Qpid
Issue Type: Bug
Components: Java Broker
Affects Versions: qpid-java-6.0
Reporter: Alex Rudyy
Priority: Blocker
Fix For: qpid-java-6.0
Unexpected number of of instances of NonPooledByteBufferRef is created on
performing destruction test to exercise memory usage on publishing and
consuming messages with already enqueued large number of persistent messages
exceeding its memory capacity in patterns that cause both some messages to pass
almost immediately though the broker, and others to be retained over a larger
period of time before consumption.
# I started broker with default memory settings (512M of Heap and 1.5G of
direct memory), published 290000 of 20k persistent messages using
StressTestClient and verified that memory heap usage reached the limit and
messages started to flow to disk:
{noformat}
java -Xmx500m -cp "..." -Dqpid.dest_syntax=BURL -Dqpid.amqp.version=0-9
org.apache.qpid.tools.StressTestClient messagecount=290000 size=20000
persistent=true consumers=0 random=false reportmod=1000
{noformat}
# Broker heap dump was collected after full GC.
# Two other StressTestClient processes have been started: one publishing into
queue with a lot of messages and another publishing into empty queue:
{noformat}
java -Xmx500m -cp "..." -Dqpid.dest_syntax=BURL -Dqpid.amqp.version=0-9
org.apache.qpid.tools.StressTestClient messagecount=10 size=20000
persistent=true random=false repetitions=300 lowprefetch=true
{noformat}
{noformat}
java -Xmx500m -cp "..." -Dqpid.dest_syntax=BURL -Dqpid.amqp.version=0-9
org.apache.qpid.tools.StressTestClient messagecount=10 size=20000
persistent=true random=true suffix=_b repetitions=300 lowprefetch=true
{noformat}
# Broker heap dump was collected after full GC
# Repeated 2 steps above 3 more times
What I noticed in the heap dump that on running steps 3-4 during 3 first
iterations the number of NonPooledByteBufferRef was growing by ~3000 in each
iteration. Only on last 4th iterations the number of NonPooledByteBufferRef
increased on ~1000 (which indicates that GC started to collect old
NonPooledByteBufferRef).
The increase in number of NonPooledByteBufferRef on each test iteration looks
suspicious and requires further investigation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
