[
https://issues.apache.org/jira/browse/QPID-3157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robbie Gemmell updated QPID-3157:
---------------------------------
Description:
subscriptions 'removed' from a queue subscription list are only marked deleted
and can not actually bereleased from memory until the head of the subscription
list advances beyond them. Additionally and perhaps more troublesome, a queues
'lastSubscriptionNode' can refer to a particular subscription but hold *all*
subscequent subscriptions in memory whether they have been deleted or not and
regardless whether the head of the SubscriptionList has advanced beyond them,
As a result any memory in use by the now-closed subscription will not be
released until the queue is deleted, or all the subscriptions prior to it are
closed and removed from the list *and* the 'lastSubscriptionNode' advances
beyond them.. This also holds the associated ServerSession in memory, which is
currently a rather heavyweight object.
This issue is further compounded when the queue has a backlog of messages and
consumers are then created, used to recieve one message, and closed. In this
scenario the broker sends the client as many messages as it can prefetch,
leading to creation of up to <prefetch size, default=500> MessageTransfer
commands, all which are recorded in the ServerSession but then left retained as
'unprocessed' in the closed session, which might be held in memory as described
above. This results in an explosion in the number of MessageTransfer commands
retained in memory as each message can have up to <prefetch size>
MessageTransfer commands associated with it before it is eventually consumed by
a client and all of thse will also be retained in memory by a 'removed' but not
deleted subscription. The last effect can be combated by restricting the
preftech size (eg appending &maxprefetch='1' to the connection url).
was:
subscriptions 'removed' from a queue subscription list are only marked deleted
and not actually released from memory. As a result any memory in use by the
now-closed subscription will not be released until the queue is deleted. This
also holds the ServerSession in memory.
This issue compounded when the queue has a backlog of messages and consumers
are created, used to recieve one message, and then closed. In this scenario the
broker sends the client as many messages as it can prefetch, leading to
creation of up to <prefetch size, default=500> MessageTransfer commands, all
which are recorded in the ServerSession but then left retained as 'unprocessed'
in the closed session. This results in an explosion in the number of
MessageTransfer commands retained in memory as each message can have up to
<prefetch size> MessageTransfer commands associated with it before it is
eventually consumed by a client and all of thse will also be retained in
memory. The last effect can be combated by restricting the preftech size (eg
appending &maxprefetch='1' to the connection url).
Affects Version/s: 0.10
Fix Version/s: (was: 0.10)
0.11
Summary: 'removed' subscriptions may be held in memory by the
queues SubscriptionList or 'lastSubscriptionNode' reference (was:
subscriptions 'removed' from a queue subscription list are only marked deleted
and not actually completely removed from the list)
Updating this with additional details from further analysis, and bumping out to
0.11: none of these issues are new to 0.10 (its been this way since time began
apparently) and the number of aspects involved here make this too sensitive to
introduce changes to so late in the 0.10 release process.
> 'removed' subscriptions may be held in memory by the queues SubscriptionList
> or 'lastSubscriptionNode' reference
> ----------------------------------------------------------------------------------------------------------------
>
> Key: QPID-3157
> URL: https://issues.apache.org/jira/browse/QPID-3157
> Project: Qpid
> Issue Type: Bug
> Components: Java Broker
> Affects Versions: 0.5, 0.6, 0.7, 0.8, 0.9, 0.10
> Reporter: Robbie Gemmell
> Priority: Blocker
> Fix For: 0.11
>
>
> subscriptions 'removed' from a queue subscription list are only marked
> deleted and can not actually bereleased from memory until the head of the
> subscription list advances beyond them. Additionally and perhaps more
> troublesome, a queues 'lastSubscriptionNode' can refer to a particular
> subscription but hold *all* subscequent subscriptions in memory whether they
> have been deleted or not and regardless whether the head of the
> SubscriptionList has advanced beyond them,
> As a result any memory in use by the now-closed subscription will not be
> released until the queue is deleted, or all the subscriptions prior to it are
> closed and removed from the list *and* the 'lastSubscriptionNode' advances
> beyond them.. This also holds the associated ServerSession in memory, which
> is currently a rather heavyweight object.
> This issue is further compounded when the queue has a backlog of messages and
> consumers are then created, used to recieve one message, and closed. In this
> scenario the broker sends the client as many messages as it can prefetch,
> leading to creation of up to <prefetch size, default=500> MessageTransfer
> commands, all which are recorded in the ServerSession but then left retained
> as 'unprocessed' in the closed session, which might be held in memory as
> described above. This results in an explosion in the number of
> MessageTransfer commands retained in memory as each message can have up to
> <prefetch size> MessageTransfer commands associated with it before it is
> eventually consumed by a client and all of thse will also be retained in
> memory by a 'removed' but not deleted subscription. The last effect can be
> combated by restricting the preftech size (eg appending &maxprefetch='1' to
> the connection url).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:[email protected]
