No problems, Gary, and I appreciate your input. I've created https://issues.apache.org/jira/browse/ARTEMIS-3693
-----Original Message----- From: Gary Tully <gary.tu...@gmail.com> Sent: Thursday, 17 February 2022 9:53 PM To: users@activemq.apache.org Subject: Re: Artemis duplicate detection apologies, I must have been dreaming of how it should be! my bad. I know there were multiple cases where it needed to be disabled for protocols. the only other thing that is left is to set the duplicate id cache size to 1, it does not support any value <= 0. that is broker wide however, so not configurable at the address level, it can maybe work ok for transaction batches > 1, but an immediate resend of the same message will be rejected. id-cache-size = 1 in broker.xml as referenced by https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fduplicate-detection.html&data=04%7C01%7CBrad.Harvey%40gbst.com%7C75d95f7ad3394440777b08d9f20c0c7a%7C1c2da354196b481891e4f760cbaac9e4%7C0%7C0%7C637806955849663699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=pyW2v8wEhkA6VU%2BZk4S4hyBiY6DzkEfAJI4lmWk1HaY%3D&reserved=0 Would you mind raising a JIRA issue with your use case and wish as to how it should be configured? Really just capturing this email thread. On Thu, 17 Feb 2022 at 09:59, Brad Harvey <brad.har...@gbst.com> wrote: > > Hi Gary, > > Thanks for the tips. Turning off the duplicate detection at the address > level sounds like what I'm after, but I couldn't spot an applicable setting > at the address level - only in the bridge/cluster connections. > > I looked in > https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Facti > vemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fdupl > icate-detection.html&data=04%7C01%7CBrad.Harvey%40gbst.com%7C75d95 > f7ad3394440777b08d9f20c0c7a%7C1c2da354196b481891e4f760cbaac9e4%7C0%7C0 > %7C637806955849663699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ > IjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=pyW2v8wEhk > A6VU%2BZk4S4hyBiY6DzkEfAJI4lmWk1HaY%3D&reserved=0 > https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Facti > vemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fconf > iguration-index.html&data=04%7C01%7CBrad.Harvey%40gbst.com%7C75d95 > f7ad3394440777b08d9f20c0c7a%7C1c2da354196b481891e4f760cbaac9e4%7C0%7C0 > %7C637806955849663699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ > IjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=pzoTS7kNhi > 63ruTNLQ8Q9wMRUH3Zt2iguKpSW8TyhgQ%3D&reserved=0 > https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith > ub.com%2Fapache%2Factivemq-artemis%2Fblob%2Fmain%2Fartemis-server%2Fsr > c%2Fmain%2Fresources%2Fschema%2Fartemis-configuration.xsd&data=04% > 7C01%7CBrad.Harvey%40gbst.com%7C75d95f7ad3394440777b08d9f20c0c7a%7C1c2 > da354196b481891e4f760cbaac9e4%7C0%7C0%7C637806955849663699%7CUnknown%7 > CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV > CI6Mn0%3D%7C3000&sdata=W2R%2B76d8hU9DXKUuQS02Ek1Ar31lhImbjtZ9v5lLQ > Ko%3D&reserved=0 > > Thanks > Brad > > -----Original Message----- > From: Gary Tully <gary.tu...@gmail.com> > Sent: Wednesday, 16 February 2022 8:45 PM > To: users@activemq.apache.org > Subject: Re: Artemis duplicate detection > > The fix in > https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FARTEMIS-3521&data=04%7C01%7CBrad.Harvey%40gbst.com%7C75d95f7ad3394440777b08d9f20c0c7a%7C1c2da354196b481891e4f760cbaac9e4%7C0%7C0%7C637806955849663699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=e82%2FjCdu6zn9eBqZTA%2FapCYpoJvBXI5GKFjT40pCEMM%3D&reserved=0 > gives you the option to disable the use of duplicate id by the openwire > protocol adapter. > see example usage in the test: > https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitb > ox.apache.org%2Frepos%2Fasf%3Fp%3Dactivemq-artemis.git%3Ba%3Dblob%3Bf% > 3Dtests%2Fintegration-tests%2Fsrc%2Ftest%2Fjava%2Forg%2Fapache%2Factiv > emq%2Fartemis%2Ftests%2Fintegration%2Fopenwire%2FFailoverDuplicateIDUs > ageTest.java%3Bh%3D3231256e1a5d037d364844221813ef51392eb02e%3Bhb%3D15c > 2dbb%23l54&data=04%7C01%7CBrad.Harvey%40gbst.com%7C75d95f7ad339444 > 0777b08d9f20c0c7a%7C1c2da354196b481891e4f760cbaac9e4%7C0%7C0%7C6378069 > 55849663699%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMz > IiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=dbHqoOZ6MjSxKPijwBd4 > m9rPVz96Q36Mdbnsq08Eb0M%3D&reserved=0 > > Otherwise, the use of duplicate detection is configurable on an address level > via address settings, you could disable it altogether with a wildcard match. > > On Wed, 16 Feb 2022 at 08:08, Brad Harvey <brad.har...@gbst.com> wrote: > > > > Hi, > > > > I'll put my questions up the top so they don't get lost, see background > > below for the context. > > > > Question 1 - I do not want duplicate detection (1) to roll back entire > > transactions. Is there a way to change the behaviour so that the entire > > transaction does not get rolled back? The behaviour I would like is to > > either silently drop the single duplicate message and commit the rest of > > the transaction, or not be enabled at all (as the application has duplicate > > detection). > > > > Question 2 - I think consuming a message from artemis using the openwire > > protocol sets the _AMQ_DUPL_ID header on the consumed message which is > > available to the client to read (2). Is there anything in the activemq > > classic openwire or qpid jms amqp libraries that would result in this > > property being set under the covers when sending a message? > > > > > > Background: > > > > I ran into issue where artemis' duplicate detection triggered, causing > > transaction rollback. I say unexpected because we don't set the duplicate > > detection header explicitly in our code. > > > > Artemis 2.20.0. Communication is using the Openwire protocol with ActiveMQ > > Classic JMS client libraries. > > The application is Apache Camel based. There's a route that reads multiple > > messages from queue "A", does "stuff", then sends them onto another queue > > "B" using a JMS transacted session. > > > > The problem encountered was a failure in committing the transaction after > > writing to queue "B": > > > > 2022-02-14 12:52:16.898 INFO ||ID-syn-ang-rbau-csp-1a-1644803524964-0-40| > > 1593489 --- [n.FromCSPSigned] org.apache.activemq.TransactionContext : > > commit failed for transaction TX:ID:hostname-44825-1644803526098-1:1:4 > > javax.jms.JMSException: > > org.apache.activemq.artemis.api.core.ActiveMQDuplicateIdException: > > Duplicate message detected - message will not be routed. Message > > information:CoreMessage[ <snip> > > _AMQ_DUPL_ID=ID:hostname1-42351-1643245903980-1:132:2746:5:1, <snip> > > > > This caused a rollback & retry in the camel route (expected) which > > continued to fail with the same exception until retries were exceeded and > > the batch of messages was moved from queue "A" to the DLQ. > > > > I believe the way the header got onto the message being sent was: > > 1. When sending messages to an openwire consumer that uses the > > failover transport, artemis adds the _AMQ_DUPL_ID header to the message. > > (2) 2. Camel copies all JMS headers to camel message headers when consuming > > a JMS message. > > 3. Camel copies all camel messages headers to JMS headers when sending a > > JMS message. > > > > And so #3 dutifully set the _AMQ_DUPL_ID header. I think the solution is > > for our camel application to use header filters on the camel components so > > that #2/#3 does not occur. But I'm concerned about other ways this header > > might get set - hence question 1 and 2 at the start of the email. > > > > Thanks for your time and guidance. > > Brad Harvey > > > > (1) > > https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fac > > ti > > vemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fdu > > pl > > icate-detection.html&data=04%7C01%7CBrad.Harvey%40gbst.com%7Cead > > 9b > > fed28a7438968c508d9f1397115%7C1c2da354196b481891e4f760cbaac9e4%7C0%7 > > C0 > > %7C637806051335437239%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLC > > JQ > > IjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=UQd3Bfe5 > > Jl > > 3GNmZQXGy%2BRTCL54WY829sBeqpdOM%2BGZA%3D&reserved=0 > > (2) > > https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi > > th > > ub.com%2Fapache%2Factivemq-artemis%2Fblob%2F1e6297957766138c312273cb > > bd > > 15327c22f91c53%2Fartemis-protocols%2Fartemis-openwire-protocol%2Fsrc > > %2 > > Fmain%2Fjava%2Forg%2Fapache%2Factivemq%2Fartemis%2Fcore%2Fprotocol%2 > > Fo > > penwire%2Famq%2FAMQSession.java%23L397&data=04%7C01%7CBrad.Harve > > y% > > 40gbst.com%7Cead9bfed28a7438968c508d9f1397115%7C1c2da354196b481891e4 > > f7 > > 60cbaac9e4%7C0%7C0%7C637806051335437239%7CUnknown%7CTWFpbGZsb3d8eyJW > > Ij > > oiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000& > > am > > p;sdata=RZLPGmOSk%2BGAL5AGjs2wH39vz%2B%2Bg5XQc5bzzzJf5dpo%3D&res > > er > > ved=0 > > > > The information transmitted is intended only for the person or entity to > > which it is addressed and may contain confidential and / or privileged > > material that may be governed by confidential information provisions > > contained in the agreement between GBST and your company. Any disclosure, > > copying, distribution, or other use without the express consent of the > > sender is prohibited. If you received this in error, please contact the > > sender and delete the material from any computer. All rights in the > > information transmitted, including copyright, are reserved. Nothing in this > > message should be interpreted as a digital signature that can be used to > > authenticate a document. No warranty is given by the sender that any > > attachments to this email are free from viruses or other defects. > The information transmitted is intended only for the person or entity to > which it is addressed and may contain confidential and / or privileged > material that may be governed by confidential information provisions > contained in the agreement between GBST and your company. Any disclosure, > copying, distribution, or other use without the express consent of the sender > is prohibited. If you received this in error, please contact the sender and > delete the material from any computer. All rights in the information > transmitted, including copyright, are reserved. Nothing in this message > should be interpreted as a digital signature that can be used to authenticate > a document. No warranty is given by the sender that any attachments to this > email are free from viruses or other defects. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and / or privileged material that may be governed by confidential information provisions contained in the agreement between GBST and your company. Any disclosure, copying, distribution, or other use without the express consent of the sender is prohibited. If you received this in error, please contact the sender and delete the material from any computer. All rights in the information transmitted, including copyright, are reserved. Nothing in this message should be interpreted as a digital signature that can be used to authenticate a document. No warranty is given by the sender that any attachments to this email are free from viruses or other defects.
