[
https://issues.apache.org/jira/browse/ARTEMIS-699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15473084#comment-15473084
]
Michael Brown commented on ARTEMIS-699:
---------------------------------------
Our use case is that we have hundreds of thousands of records arriving (small
load) where they form small groups of two or three that are related. For
efficiency and to reduce the risk of race hazards we use that grouping
identifier so that related records are highly likely to be processed by the
same consumer. Naturally that means that in a day we can have tens of thousands
of groups, or more. This is an initial estimate, we have to assume much higher
figures in the future.
Reaping provided a nice way to manage the number of groups, and also to manage
other issues such rebalancing when new consumers come online. Since we only
needed it to be likely to target the same node, it was ok that there is a risk
of a group switching nodes.
Your sibling project ActiveMQ has a nice solution for the issue of high group
counts, which is to use hash buckets. It would be a very minor change to
Artemis to support this, and shouldn't effect the low group count use cases at
all.
To quote: "How Message Groups work
When a message is being dispatched to a consumer, the JMSXGroupID is checked.
If one is present then the broker checks to see if a consumer owns that message
group. (Since there could be a massive number of individual message groups we
use *hash buckets* rather than the actual JMSXGroupID string)."
I hope that you don't think it an imposition if I raise a ticket to do
something similar.
> Reaper seems to clog up the journal, and messages fail to deliver
> -----------------------------------------------------------------
>
> Key: ARTEMIS-699
> URL: https://issues.apache.org/jira/browse/ARTEMIS-699
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
> Affects Versions: 1.3.0
> Reporter: Michael Brown
> Assignee: Justin Bertram
> Priority: Critical
> Fix For: 1.4.0
>
> Attachments: clustered-grouping.tar.gz
>
>
> We are seeing the exception (below) regularly, and a loss of message
> delivery, which only goes away if we delete the journal and restart. This is
> with the following grouping options in broker.xml:
> {code:xml}
> <grouping-handler name="jms-recordsure-grouping-handler">
> <type>LOCAL</type>
> <address>jms</address>
> <timeout>5000</timeout>
> <group-timeout>300000</group-timeout>
> <reaper-period>30000</reaper-period>
> </grouping-handler>
> {code}
> If we remove timeout/group-timeout/reaper-period the issues go away.
> Having traced through the code I see this issue in LocalGroupHandling.java:
> {code:java}
> if (expiredGroups >= 1000 && txID >= 0) {
> expiredGroups = 0;
> txID = -1;
> storageManager.commitBindings(txID);
> }
> {code}
> The method 'commitBindings' has (via dependencies) an explicit check that
> will throw an exception if txID is -1. This code must fail.
> I suggest a fix that moves the reset of txID to be after the commitBindings
> call, as follows:
> {code:java}
> if (expiredGroups >= 1000 && txID >= 0) {
> storageManager.commitBindings(txID);
> expiredGroups = 0;
> txID = -1;
> }
> {code}
> {noformat}
> 2016-08-24 13:09:01,660 WARN [org.apache.activemq.artemis.core.server]
> AMQ222114: Unable to delete group binding info
> c2e95857-87ad-455a-ba67-1c63ac351cc8.ad900ba9-69e3-11e6-86b2-0242ac110003:
> java.lang.IllegalStateException: Cannot find tx with id -1
> at
> org.apache.activemq.artemis.core.journal.impl.JournalImpl.appendCommitRecord(JournalImpl.java:1013)
> [artemis-journal-1.3.0.jar:1.3.0]
> at
> org.apache.activemq.artemis.core.journal.impl.JournalBase.appendCommitRecord(JournalBase.java:112)
> [artemis-journal-1.3.0.jar:1.3.0]
> at
> org.apache.activemq.artemis.core.journal.impl.JournalImpl.appendCommitRecord(JournalImpl.java:79)
> [artemis-journal-1.3.0.jar:1.3.0]
> at
> org.apache.activemq.artemis.core.replication.ReplicatedJournal.appendCommitRecord(ReplicatedJournal.java:164)
> [artemis-server-1.3.0.jar:1.3.0]
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.commitBindings(AbstractJournalStorageManager.java:650)
> [artemis-server-1.3.0.jar:1.3.0]
> at
> org.apache.activemq.artemis.core.server.group.impl.LocalGroupingHandler$GroupIdReaper.run(LocalGroupingHandler.java:449)
> [artemis-server-1.3.0.jar:1.3.0]
> at
> org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:103)
> [artemis-core-client-1.3.0.jar:1.3.0]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [rt.jar:1.8.0_102]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [rt.jar:1.8.0_102]
> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_102]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
