[
https://issues.apache.org/jira/browse/ARTEMIS-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17723810#comment-17723810
]
Liviu Citu edited comment on ARTEMIS-4276 at 5/18/23 7:23 AM:
--------------------------------------------------------------
*Virtual Topics vs Shared Topic Consumers*
Our plan during migration from *Classic ActiveMQ* to *Artemis* is to modify as
little as possible the source code to reduce the regression impact.
Our software is C++ code based and we are using *CMS API* ({*}ActiveMQ CPP{*})
as a client. I am unable to find a CMS API to create shared topic consumer so I
am not sure if it exists. In the same time, I am not very sure that the
behavior of such LB using shared subscription is what we want in our Gateway
Loader Servers. We do not want to process the same message in more than one
group (please correct me if I am wrong):
[http://jmesnil.net/weblog/2013/06/27/jms-20-shared-subscription/]
Nonetheless we were using virtual topics with Classic ActiveMQ and they work as
expected with Artemis too (the setup changes are trivial).
*Idempotent consumer using local, volatile LRU cache*
*ActiveMQ CPP* does not have idempotent consumers. Nonetheless in our software
we have a wrapper over the CMS consumer and a wrapper over the CMS consumer
listener. The LRU cache is part of our listener. Indeed the *CMS consumer* gets
restored during failover but the object is not recreated so our wrapper is
still valid and the cache still stands in this context. Indeed this might not
be the best option to handle the duplicated messages but when there is no Load
Balance it works ok. The problem is indeed when there are more than one
consumer involved for the same topic.
*XA transaction*
The synchronization problem between database and JMS Broker is not necessary
related to failover or Artemis usage. We have this also with Classic ActiveMQ
[for instance if there is a network glitch or when ActiveMQ Broker goes down
and the message reached the database]. We were exploring the usage of XA
transaction however the code changes needed to implement it in an existing
software is huge and practically impossible.
*Handle duplicate messages at database level*
At the database level we have a protection with primary keys and indeed the
same transaction cannot be inserted twice. The problem with this is more like a
user notification problem.
As I have explained in the description of this issue, we have also a Gateway
Fail Queue Monitor where the users might find all messages that failed during
processing (included those duplicated that failed during insertion).
We just wanted to explore the possibility to have a way of removing these
"fake" failures caused by failover or somehow to distinguish them from those
which are real business failures. These are technical failures (cause by
failover in this case) and users looking to the Fail Queue Monitor might get
confused when seeing such duplicated messages without understanding what went
wrong (if indeed the same duplicated transaction was received from external
system of the duplicated message is caused by failover). I suppose they will
have to deal with this as being a system limitation.
was (Author: JIRAUSER300236):
*Virtual Topics vs Shared Topic Consumers*
Our plan during migration from *Classic ActiveMQ* to *Artemis* is to modify as
little as possible the source code to reduce the regression impact.
Our software is C++ code based and we are using *CMS API* ({*}ActiveMQ CPP{*})
as a client. I am unable to find a CMS API to create shared topic consumer so I
am not sure if it exists. In the same time, I am not very sure that the
behavior of such LB using shared subscription is what we want in our Gateway
Loader Servers. We do not want to process the same message in more than one
group (please correct me if I am wrong):
[http://jmesnil.net/weblog/2013/06/27/jms-20-shared-subscription/]
Nonetheless we were using virtual topics with Classic ActiveMQ and they work as
expected with Artemis too (the setup changes are trivial).
*Idempotent consumer using local, volatile LRU cache*
*ActiveMQ CPP* does not have idempotent consumers. Nonetheless in our software
we have a wrapper over the CMS consumer and a wrapper over the CMS consumer
listener. The LRU cache is part of our listener. Indeed the *CMS consumer* gets
restored during FailOver but the object is not recreated so our wrapper is
still valid and the cache still stands in this context. Indeed this might not
be the best option to handle the duplicated messages but when there is no Load
Balance it works ok. The problem is indeed when there are more than one
consumer involved for the same topic.
*XA transaction*
The synchronization problem between database and JMS Broker is not necessary
related to FailOver or Artemis usage. We have this also with Classic ActiveMQ
[for instance if there is a network glitch or when *ActiveMQ* goes down and the
message reached the database]. We were exploring the usage of XA transaction
however the code changes needed to implement it in an existing software is huge
and practically impossible.
*Handle duplicate messages at database level*
At the database level we have a protection with primary keys and indeed the
same transaction cannot be inserted twice. The problem with this is more like a
user notification problem.
As I have explained in the description of this issue, we have also a Gateway
Fail Queue Monitor where the users might find all messages that failed during
processing (included those duplicated that failed during insertion).
We just wanted to explore the possibility to have a way of removing these
"fake" failures caused by FailOver or somehow to distinguish them from those
which are real business failures. These are technical failures (cause by
FailOver in this case) and users looking to the Fail Queue Monitor might get
confused when seeing such duplicated messages without understanding what went
wrong. I suppose they will have to deal with this as being a system limitation.
> Message Group does not replicate properly during failover
> ---------------------------------------------------------
>
> Key: ARTEMIS-4276
> URL: https://issues.apache.org/jira/browse/ARTEMIS-4276
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Affects Versions: 2.28.0
> Reporter: Liviu Citu
> Priority: Major
>
> Hi,
> We are currently migrating our software from Classic to Artemis and we plan
> to use failover functionality.
> We were using message group functionality by setting *JMSXGroupID* and this
> was working as expected. However after failover switch I noticed that
> messages are sent to wrong consumers.
> Our gateway/interface application is actually a collection of servers:
> * gateway adapter server: receives messages from an external systems and
> puts them on a specific/virtual topic
> * gateway loader server (can be balanced): picks up the messages from the
> topic and do processing
> * gateway fail queue: monitors all messages that failed processing and has a
> functionality of resubmitting the message (users will correct the processing
> errors and then resubmit transaction)
> *JMSXGroupID* is used to ensure that during message resubmit the same
> consumer/loader is processing the message as it was originally processed.
> However, if the message resubmit is happening during failover switch we have
> noticed that the message is not sent to the right consumer as it should.
> Basically the first available consumer is used which is not what we want.
> I have searched for configuration changes but couldn't find any relevant
> information.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
