[
https://issues.apache.org/jira/browse/ARTEMIS-5773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18049953#comment-18049953
]
Justin Bertram commented on ARTEMIS-5773:
-----------------------------------------
bq. The the memory consumption from #6085 is more efficient. See the attached
charts.
Were both those graphs generated from the exact same use-case (i.e. same data
in & out)? I'm surprised the results are _so_ different. I would expect some
variation based on the timing of the reaper, but this seems like something
categorically different. That said, in your reproducer you're passing {{500}}
to
{{org.apache.activemq.artemis.core.config.Configuration#setAddressQueueScanPeriod}}
whereas you're passing {{5000}} to
{{org.apache.activemq.artemis.core.settings.impl.AddressSettings#setAutoDeleteAddressesDelay}}.
I think these two values should be equal between the two runs in order to
align semantics and get a more fair comparison.
bq. Although the change technically works on a "greenfield" deployment, it
appears that the cleanup of caches does not work on a "brownfield" deployment.
This is expected. I think we'll need to add some additional management
capabilities around the duplicate ID caches so that folks can clean these up
manually.
bq. I have augmented artemis-leak-repro to demonstrate the issue.
Were you referring to the greenfield vs. brownfield issue or the disparity in
the metrics or both?
To be clear, I think clustering here is kind of a red herring. The general
issue (i.e. not clearing the duplicate ID cache when the address is deleted)
can be reproduced without using clustering at all. It may be worth simplifying
your reproducer to take this into account.
> Memory and storage leak in PostOfficeImpl and PagingManagerImpl
> ---------------------------------------------------------------
>
> Key: ARTEMIS-5773
> URL: https://issues.apache.org/jira/browse/ARTEMIS-5773
> Project: Artemis
> Issue Type: Bug
> Affects Versions: 2.44.0
> Reporter: Scott Feldstein
> Assignee: Justin Bertram
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.50.0
>
> Attachments: Testing post #6153.png, storage-freed-after-patch.png
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> h1. Problem
> We are encountering memory leaks in {{PostOfficeImpl}} and
> {{PagingManagerImpl}}. We have a clustered set of nodes and use the BRIDGE
> queues to transmit messages between the nodes. We use > 10_000 topics where
> the ActiveMQServer Configuration is setup to auto-delete them. When this
> occurs the BRIDGE addresses are not purged. This causes bloating in the
> journal files and in memory. The memory issue occurs in the
> {{PagingManagerImpl.stores}} and {{PostOfficeImpl.duplicateIDCaches}} (see
> below).
> From the analysis I see these orphaned addresses need to be cleaned up. The
> periodic Reaper process needs to be augmented to achieve this.
> h1. Report from Memory Analyzer
> {code}
> Class Name
> | Shallow Heap | Retained Heap | Percentage
> ---------------------------------------------------------------------------------------------------------------------------------------------------
> org.apache.activemq.artemis.core.paging.impl.PagingManagerImpl @ 0x704d4dd40
> | 0.10 KB | 296,632.09 KB | 15.60%
> |- stores java.util.concurrent.ConcurrentHashMap @ 0x704d4de70
> | 0.06 KB | 296,631.45 KB | 15.60%
> |- syncLock java.util.concurrent.locks.ReentrantReadWriteLock @ 0x704d4dda8
> | 0.02 KB | 0.12 KB | 0.00%
> |- managerExecutor org.apache.activemq.artemis.utils.actors.OrderedExecutor @
> 0x7311d2e10 | 0.04 KB | 0.12 KB | 0.00%
> |- globalSizeMetric org.apache.activemq.artemis.utils.SizeAwareMetric @
> 0x7311d2e88 | 0.08 KB | 0.11 KB | 0.00%
> |- blockedStored
> org.apache.activemq.artemis.utils.collections.ConcurrentHashSet @ 0x704d4de20
> | 0.02 KB | 0.08 KB | 0.00%
> |- transactions java.util.concurrent.ConcurrentHashMap @ 0x7311d2da0
> | 0.06 KB | 0.06 KB | 0.00%
> |- memoryCallback java.util.concurrent.ConcurrentLinkedQueue @ 0x7311d2de0
> | 0.02 KB | 0.05 KB | 0.00%
> '- Total: 7 entries
> | | |
> org.apache.activemq.artemis.core.postoffice.impl.PostOfficeImpl @ 0x704d20ec0
> | 0.09 KB | 272,058.85 KB | 14.31%
> |- duplicateIDCaches java.util.concurrent.ConcurrentHashMap @ 0x75da26050
> | 0.06 KB | 205,443.41 KB | 10.80%
> |- addressManager
> org.apache.activemq.artemis.core.postoffice.impl.WildcardAddressManager @
> 0x75da26090| 0.05 KB | 66,605.27 KB | 3.50%
> |- queueInfos java.util.HashMap @ 0x7311d0408
> | 0.05 KB | 8.81 KB | 0.00%
> |- org.apache.activemq.artemis.api.core.SimpleString @ 0x72325d418
> | 0.03 KB | 0.41 KB | 0.00%
> |- org.apache.activemq.artemis.api.core.SimpleString @ 0x72325d7d0
> | 0.03 KB | 0.41 KB | 0.00%
> |- org.apache.activemq.artemis.api.core.SimpleString @ 0x72325d6d0
> | 0.03 KB | 0.25 KB | 0.00%
> |- org.apache.activemq.artemis.api.core.SimpleString @ 0x72325da70
> | 0.03 KB | 0.20 KB | 0.00%
> |- notificationLock java.lang.Object @ 0x75da26040
> | 0.02 KB | 0.02 KB | 0.00%
> '- Total: 8 entries
> | | |
> ---------------------------------------------------------------------------------------------------------------------------------------------------
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
