[
https://issues.apache.org/jira/browse/ARTEMIS-5773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Bertram updated ARTEMIS-5773:
------------------------------------
Description:
h1. Problem
We are encountering memory leaks in {{PostOfficeImpl}} and
{{PagingManagerImpl}}. We have a clustered set of nodes and use the BRIDGE
queues to transmit messages between the nodes. We use > 10_000 topics where
the ActiveMQServer Configuration is setup to auto-delete them. When this
occurs the BRIDGE addresses are not purged. This causes bloating in the
journal files and in memory. The memory issue occurs in the
{{PagingManagerImpl.stores}} and {{PostOfficeImpl.duplicateIDCaches}} (see
below).
>From the analysis I see these orphaned addresses need to be cleaned up. The
>periodic Reaper process needs to be augmented to achieve this.
h1. Report from Memory Analyzer
{code}
Class Name
| Shallow Heap | Retained Heap | Percentage
---------------------------------------------------------------------------------------------------------------------------------------------------
org.apache.activemq.artemis.core.paging.impl.PagingManagerImpl @ 0x704d4dd40
| 0.10 KB | 296,632.09 KB | 15.60%
|- stores java.util.concurrent.ConcurrentHashMap @ 0x704d4de70
| 0.06 KB | 296,631.45 KB | 15.60%
|- syncLock java.util.concurrent.locks.ReentrantReadWriteLock @ 0x704d4dda8
| 0.02 KB | 0.12 KB | 0.00%
|- managerExecutor org.apache.activemq.artemis.utils.actors.OrderedExecutor @
0x7311d2e10 | 0.04 KB | 0.12 KB | 0.00%
|- globalSizeMetric org.apache.activemq.artemis.utils.SizeAwareMetric @
0x7311d2e88 | 0.08 KB | 0.11 KB | 0.00%
|- blockedStored
org.apache.activemq.artemis.utils.collections.ConcurrentHashSet @ 0x704d4de20
| 0.02 KB | 0.08 KB | 0.00%
|- transactions java.util.concurrent.ConcurrentHashMap @ 0x7311d2da0
| 0.06 KB | 0.06 KB | 0.00%
|- memoryCallback java.util.concurrent.ConcurrentLinkedQueue @ 0x7311d2de0
| 0.02 KB | 0.05 KB | 0.00%
'- Total: 7 entries
| | |
org.apache.activemq.artemis.core.postoffice.impl.PostOfficeImpl @ 0x704d20ec0
| 0.09 KB | 272,058.85 KB | 14.31%
|- duplicateIDCaches java.util.concurrent.ConcurrentHashMap @ 0x75da26050
| 0.06 KB | 205,443.41 KB | 10.80%
|- addressManager
org.apache.activemq.artemis.core.postoffice.impl.WildcardAddressManager @
0x75da26090| 0.05 KB | 66,605.27 KB | 3.50%
|- queueInfos java.util.HashMap @ 0x7311d0408
| 0.05 KB | 8.81 KB | 0.00%
|- org.apache.activemq.artemis.api.core.SimpleString @ 0x72325d418
| 0.03 KB | 0.41 KB | 0.00%
|- org.apache.activemq.artemis.api.core.SimpleString @ 0x72325d7d0
| 0.03 KB | 0.41 KB | 0.00%
|- org.apache.activemq.artemis.api.core.SimpleString @ 0x72325d6d0
| 0.03 KB | 0.25 KB | 0.00%
|- org.apache.activemq.artemis.api.core.SimpleString @ 0x72325da70
| 0.03 KB | 0.20 KB | 0.00%
|- notificationLock java.lang.Object @ 0x75da26040
| 0.02 KB | 0.02 KB | 0.00%
'- Total: 8 entries
| | |
---------------------------------------------------------------------------------------------------------------------------------------------------
{code}
was:
h1. Problem
We are encountering memory leaks in PostOfficeImpl and PagingManagerImpl. We
have a clustered set of nodes and use the BRIDGE queues to transmit messages
between the notes. We use > 10_000 topics where the ActiveMQServer
Configuration is setup to auto-delete them. When this occurs the BRIDGE
addresses are not purged. This causes bloating in the journal files and in
memory. The memory issue occurs in the PagingManagerImpl.stores and
PostOfficeImpl.duplicateIDCaches (see below).
>From the analysis I see these orphaned addresses need to be cleaned up. The
>periodic Reaper process needs to be augmented to achieve this.
h1. Report from Memory Analyzer
{code}
Class Name
| Shallow Heap | Retained Heap | Percentage
---------------------------------------------------------------------------------------------------------------------------------------------------
org.apache.activemq.artemis.core.paging.impl.PagingManagerImpl @ 0x704d4dd40
| 0.10 KB | 296,632.09 KB | 15.60%
|- stores java.util.concurrent.ConcurrentHashMap @ 0x704d4de70
| 0.06 KB | 296,631.45 KB | 15.60%
|- syncLock java.util.concurrent.locks.ReentrantReadWriteLock @ 0x704d4dda8
| 0.02 KB | 0.12 KB | 0.00%
|- managerExecutor org.apache.activemq.artemis.utils.actors.OrderedExecutor @
0x7311d2e10 | 0.04 KB | 0.12 KB | 0.00%
|- globalSizeMetric org.apache.activemq.artemis.utils.SizeAwareMetric @
0x7311d2e88 | 0.08 KB | 0.11 KB | 0.00%
|- blockedStored
org.apache.activemq.artemis.utils.collections.ConcurrentHashSet @ 0x704d4de20
| 0.02 KB | 0.08 KB | 0.00%
|- transactions java.util.concurrent.ConcurrentHashMap @ 0x7311d2da0
| 0.06 KB | 0.06 KB | 0.00%
|- memoryCallback java.util.concurrent.ConcurrentLinkedQueue @ 0x7311d2de0
| 0.02 KB | 0.05 KB | 0.00%
'- Total: 7 entries
| | |
org.apache.activemq.artemis.core.postoffice.impl.PostOfficeImpl @ 0x704d20ec0
| 0.09 KB | 272,058.85 KB | 14.31%
|- duplicateIDCaches java.util.concurrent.ConcurrentHashMap @ 0x75da26050
| 0.06 KB | 205,443.41 KB | 10.80%
|- addressManager
org.apache.activemq.artemis.core.postoffice.impl.WildcardAddressManager @
0x75da26090| 0.05 KB | 66,605.27 KB | 3.50%
|- queueInfos java.util.HashMap @ 0x7311d0408
| 0.05 KB | 8.81 KB | 0.00%
|- org.apache.activemq.artemis.api.core.SimpleString @ 0x72325d418
| 0.03 KB | 0.41 KB | 0.00%
|- org.apache.activemq.artemis.api.core.SimpleString @ 0x72325d7d0
| 0.03 KB | 0.41 KB | 0.00%
|- org.apache.activemq.artemis.api.core.SimpleString @ 0x72325d6d0
| 0.03 KB | 0.25 KB | 0.00%
|- org.apache.activemq.artemis.api.core.SimpleString @ 0x72325da70
| 0.03 KB | 0.20 KB | 0.00%
|- notificationLock java.lang.Object @ 0x75da26040
| 0.02 KB | 0.02 KB | 0.00%
'- Total: 8 entries
| | |
---------------------------------------------------------------------------------------------------------------------------------------------------
{code}
> Memory and storage leak in PostOfficeImpl and PagingManagerImpl
> ---------------------------------------------------------------
>
> Key: ARTEMIS-5773
> URL: https://issues.apache.org/jira/browse/ARTEMIS-5773
> Project: Artemis
> Issue Type: Bug
> Affects Versions: 2.44.0
> Reporter: Scott Feldstein
> Assignee: Justin Bertram
> Priority: Major
> Attachments: storage-freed-after-patch.png
>
>
> h1. Problem
> We are encountering memory leaks in {{PostOfficeImpl}} and
> {{PagingManagerImpl}}. We have a clustered set of nodes and use the BRIDGE
> queues to transmit messages between the nodes. We use > 10_000 topics where
> the ActiveMQServer Configuration is setup to auto-delete them. When this
> occurs the BRIDGE addresses are not purged. This causes bloating in the
> journal files and in memory. The memory issue occurs in the
> {{PagingManagerImpl.stores}} and {{PostOfficeImpl.duplicateIDCaches}} (see
> below).
> From the analysis I see these orphaned addresses need to be cleaned up. The
> periodic Reaper process needs to be augmented to achieve this.
> h1. Report from Memory Analyzer
> {code}
> Class Name
> | Shallow Heap | Retained Heap | Percentage
> ---------------------------------------------------------------------------------------------------------------------------------------------------
> org.apache.activemq.artemis.core.paging.impl.PagingManagerImpl @ 0x704d4dd40
> | 0.10 KB | 296,632.09 KB | 15.60%
> |- stores java.util.concurrent.ConcurrentHashMap @ 0x704d4de70
> | 0.06 KB | 296,631.45 KB | 15.60%
> |- syncLock java.util.concurrent.locks.ReentrantReadWriteLock @ 0x704d4dda8
> | 0.02 KB | 0.12 KB | 0.00%
> |- managerExecutor org.apache.activemq.artemis.utils.actors.OrderedExecutor @
> 0x7311d2e10 | 0.04 KB | 0.12 KB | 0.00%
> |- globalSizeMetric org.apache.activemq.artemis.utils.SizeAwareMetric @
> 0x7311d2e88 | 0.08 KB | 0.11 KB | 0.00%
> |- blockedStored
> org.apache.activemq.artemis.utils.collections.ConcurrentHashSet @ 0x704d4de20
> | 0.02 KB | 0.08 KB | 0.00%
> |- transactions java.util.concurrent.ConcurrentHashMap @ 0x7311d2da0
> | 0.06 KB | 0.06 KB | 0.00%
> |- memoryCallback java.util.concurrent.ConcurrentLinkedQueue @ 0x7311d2de0
> | 0.02 KB | 0.05 KB | 0.00%
> '- Total: 7 entries
> | | |
> org.apache.activemq.artemis.core.postoffice.impl.PostOfficeImpl @ 0x704d20ec0
> | 0.09 KB | 272,058.85 KB | 14.31%
> |- duplicateIDCaches java.util.concurrent.ConcurrentHashMap @ 0x75da26050
> | 0.06 KB | 205,443.41 KB | 10.80%
> |- addressManager
> org.apache.activemq.artemis.core.postoffice.impl.WildcardAddressManager @
> 0x75da26090| 0.05 KB | 66,605.27 KB | 3.50%
> |- queueInfos java.util.HashMap @ 0x7311d0408
> | 0.05 KB | 8.81 KB | 0.00%
> |- org.apache.activemq.artemis.api.core.SimpleString @ 0x72325d418
> | 0.03 KB | 0.41 KB | 0.00%
> |- org.apache.activemq.artemis.api.core.SimpleString @ 0x72325d7d0
> | 0.03 KB | 0.41 KB | 0.00%
> |- org.apache.activemq.artemis.api.core.SimpleString @ 0x72325d6d0
> | 0.03 KB | 0.25 KB | 0.00%
> |- org.apache.activemq.artemis.api.core.SimpleString @ 0x72325da70
> | 0.03 KB | 0.20 KB | 0.00%
> |- notificationLock java.lang.Object @ 0x75da26040
> | 0.02 KB | 0.02 KB | 0.00%
> '- Total: 8 entries
> | | |
> ---------------------------------------------------------------------------------------------------------------------------------------------------
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
For further information, visit: https://activemq.apache.org/contact
