[jira] [Work logged] (ARTEMIS-4588) Queue Federation and large messages move slowdown
[ https://issues.apache.org/jira/browse/ARTEMIS-4588?focusedWorklogId=904944=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-904944 ] ASF GitHub Bot logged work on ARTEMIS-4588: --- Author: ASF GitHub Bot Created on: 14/Feb/24 17:07 Start Date: 14/Feb/24 17:07 Worklog Time Spent: 10m Work Description: jbertram merged PR #4815: URL: https://github.com/apache/activemq-artemis/pull/4815 Issue Time Tracking --- Worklog Id: (was: 904944) Time Spent: 20m (was: 10m) > Queue Federation and large messages move slowdown > - > > Key: ARTEMIS-4588 > URL: https://issues.apache.org/jira/browse/ARTEMIS-4588 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker, Federation >Affects Versions: 2.30.0 >Reporter: Jean-Pascal Briquet >Assignee: Justin Bertram >Priority: Major > Attachments: federation-large-message-testcase.java > > Time Spent: 20m > Remaining Estimate: 0h > > *Artemis configuration:* > Artemis configured with a dual queue federation link on Artemis and > requirement to support via this federation link standard messages and large > messages (between 100kb and 500kb). > > *What is happening:* > When a consumer is created, one or two large messages are consumed instantly > thanks to the federation but next messages are received very slowly and > arrive one by one after a delay of 30s. > The queue federation works well with standard messages and does not have any > pause between message consumption. > The delay between each message consumed looks to be coming from the > call-timeout property set on the federation upstream, when changing its > value, it changes the pause/delay between large messages consumption. > > *Reproduction* > A test case is attached and can be run by adding it into > "tests/integration-tests/src/test/java/org/apache/activemq/artemis/tests/integration/federation/FederatedQueueTest.java" > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (ARTEMIS-3308) Federated queue will not move large messages
[
https://issues.apache.org/jira/browse/ARTEMIS-3308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17817454#comment-17817454
]
ASF subversion and git services commented on ARTEMIS-3308:
--
Commit 77d9f10a3dd3febff8fee417914806b8e0c2b4c2 in activemq-artemis's branch
refs/heads/main from Justin Bertram
[ https://gitbox.apache.org/repos/asf?p=activemq-artemis.git;h=77d9f10a3d ]
ARTEMIS-4588 30 second pause for large msgs + federation
Large message support was added to
o.a.a.a.c.s.f.FederatedQueueConsumerImpl#onMessage via cf85d35 for
ARTEMIS-3308. The problem with that change is that when onMessage
returns o.a.a.a.c.c.i.ClientConsumerImpl#callOnMessage will eventually
call o.a.a.a.c.c.i.ClientLargeMessageImpl#discardBody which eventually
ends up in o.a.a.a.c.c.i.LargeMessageControllerImpl#popPacket waiting 30
seconds (i.e. the default readTimeout) for more packets to arrive (which
never do). This happens because the FederatedQueueConsumer short-cuts
the "normal" process by using LargeMessageControllerImpl#take.
This commit fixes that by tracking the number of bytes "taken" and then
looking at that value later when discarding the body effectively
skipping the 30 second wait.
> Federated queue will not move large messages
>
>
> Key: ARTEMIS-3308
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3308
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker, Federation
>Affects Versions: 2.17.0
> Environment: Windows 10 Pro (jdk 1.8 oracle last) artemis 2.17.0
>Reporter: Alexander Andreevich Revkov
>Assignee: Gary Tully
>Priority: Major
> Fix For: 2.19.1, 2.20.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Hello. I'am create two artemis broker:
> artemis01:
> {code:java}
> artemis create --max-hops=1 artemis01{code}
> artemis02:
> {code:java}
> artemis create --max-hops=1 --port-offset=10 artemis02{code}
> user: artemis
> password: artemis
>
> In artemis01/etc/broker.xml i add:
>
> {code:java}
>
>tcp://localhost:61626
>
>
>
>
>
> artemis02-connector
>
>
>
> include-federated="false">
>
>
>
> {code}
> then run both brokers.
> I send large message to artemis02:
> {code:java}
> artemis producer --url tcp://localhost:61626 --text-size 800
> --destination TEST.FEDERATED --message-count 1 --user artemis --password
> artemis{code}
> And then try get it from artemis01:
> {code:java}
> artemis consumer --url tcp://localhost:61616 --destination TEST.FEDERATED
> --message-count 1 --user artemis --password artemis{code}
> But i can't received message. Consumer stuck it loop. And artemis01 write in
> logs many NPE:
> {code:java}
> * durable queues TEST.FEDERATED:
> - queueID=30 address:TEST.FEDERATED name:TEST.FEDERATED filter:null
> * non durable for TEST.FEDERATED:
> ..
> , direct: true, rejectDuplicates: true
> 2021-05-19 19:27:18,001 DEBUG
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ843017:
> onMessageRouteError message: ClientLargeMessageImpl[messageID=1518976,
> durable=true,
> address=TEST.FEDERATED::TEST.FEDERATED,userID=50cb6a04-b8bd-11eb-85ab-00155d7523cb,properties=TypedProperties[__AMQ_CID=50bf3501-b8bd-11eb-85ab-00155d7523cb,_AMQ_ROUTING_TYPE=1,count=0,_AMQ_LARGE_SIZE=16014077,ThreadSent=Producer
> ActiveMQQueue[TEST.FEDERATED], thread=0]], with context:
> RoutingContextImpl(Address=null, routingType=null, PreviousAddress=null
> previousRoute:null, reusable=false, version=0)
> ..
> * durable queues TEST.FEDERATED:
> - queueID=30 address:TEST.FEDERATED name:TEST.FEDERATED filter:null
> * non durable for TEST.FEDERATED:
> ..
> , direct: true, rejectDuplicates: true
> 2021-05-19 19:27:18,002 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841018: error
> routing message with ID: 1518976, exception: java.lang.NullPointerException
> {code}
>
> I see some source code and find what NPE occurs in CoreMessage.java class at
> 717 line because buffer is NULL:
> {code:java}
> buffer.setIndex(0, 0);{code}
> Not large messages works fine. Please fix it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARTEMIS-4588) Queue Federation and large messages move slowdown
[ https://issues.apache.org/jira/browse/ARTEMIS-4588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17817453#comment-17817453 ] ASF subversion and git services commented on ARTEMIS-4588: -- Commit 77d9f10a3dd3febff8fee417914806b8e0c2b4c2 in activemq-artemis's branch refs/heads/main from Justin Bertram [ https://gitbox.apache.org/repos/asf?p=activemq-artemis.git;h=77d9f10a3d ] ARTEMIS-4588 30 second pause for large msgs + federation Large message support was added to o.a.a.a.c.s.f.FederatedQueueConsumerImpl#onMessage via cf85d35 for ARTEMIS-3308. The problem with that change is that when onMessage returns o.a.a.a.c.c.i.ClientConsumerImpl#callOnMessage will eventually call o.a.a.a.c.c.i.ClientLargeMessageImpl#discardBody which eventually ends up in o.a.a.a.c.c.i.LargeMessageControllerImpl#popPacket waiting 30 seconds (i.e. the default readTimeout) for more packets to arrive (which never do). This happens because the FederatedQueueConsumer short-cuts the "normal" process by using LargeMessageControllerImpl#take. This commit fixes that by tracking the number of bytes "taken" and then looking at that value later when discarding the body effectively skipping the 30 second wait. > Queue Federation and large messages move slowdown > - > > Key: ARTEMIS-4588 > URL: https://issues.apache.org/jira/browse/ARTEMIS-4588 > Project: ActiveMQ Artemis > Issue Type: Bug > Components: Broker, Federation >Affects Versions: 2.30.0 >Reporter: Jean-Pascal Briquet >Assignee: Justin Bertram >Priority: Major > Attachments: federation-large-message-testcase.java > > Time Spent: 20m > Remaining Estimate: 0h > > *Artemis configuration:* > Artemis configured with a dual queue federation link on Artemis and > requirement to support via this federation link standard messages and large > messages (between 100kb and 500kb). > > *What is happening:* > When a consumer is created, one or two large messages are consumed instantly > thanks to the federation but next messages are received very slowly and > arrive one by one after a delay of 30s. > The queue federation works well with standard messages and does not have any > pause between message consumption. > The delay between each message consumed looks to be coming from the > call-timeout property set on the federation upstream, when changing its > value, it changes the pause/delay between large messages consumption. > > *Reproduction* > A test case is attached and can be run by adding it into > "tests/integration-tests/src/test/java/org/apache/activemq/artemis/tests/integration/federation/FederatedQueueTest.java" > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (ARTEMIS-4582) add view and update permissions to augment the manage rbac for control resources
[ https://issues.apache.org/jira/browse/ARTEMIS-4582?focusedWorklogId=904899=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-904899 ] ASF GitHub Bot logged work on ARTEMIS-4582: --- Author: ASF GitHub Bot Created on: 14/Feb/24 14:09 Start Date: 14/Feb/24 14:09 Worklog Time Spent: 10m Work Description: gtully opened a new pull request, #4820: URL: https://github.com/apache/activemq-artemis/pull/4820 …agement. security-settings Issue Time Tracking --- Worklog Id: (was: 904899) Remaining Estimate: 0h Time Spent: 10m > add view and update permissions to augment the manage rbac for control > resources > > > Key: ARTEMIS-4582 > URL: https://issues.apache.org/jira/browse/ARTEMIS-4582 > Project: ActiveMQ Artemis > Issue Type: Improvement > Components: Broker, Configuration, JMX, Web Console >Affects Versions: 2.31.0 >Reporter: Gary Tully >Assignee: Gary Tully >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > we have the manage permission that allows sending to the management address, > to access any control resource. We don't however distinguish what a user can > do. > We should segment control operations into categories: CRUD provides a basis > view for get/is (Read) > update for set or operations that mutate or modify. > We allow this sort of configuration via management.xml for jmx mbean access > but using a different model based on object name. > All of the mbeans delegate to the control resources. > If we add these two additional permissions then we can have a single rbac > model (that supports config reload) and more granularity on control resource > access from the management address. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (ARTEMIS-4582) add view and update permissions to augment the manage rbac for control resources
[ https://issues.apache.org/jira/browse/ARTEMIS-4582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Tully reassigned ARTEMIS-4582: --- Assignee: Gary Tully > add view and update permissions to augment the manage rbac for control > resources > > > Key: ARTEMIS-4582 > URL: https://issues.apache.org/jira/browse/ARTEMIS-4582 > Project: ActiveMQ Artemis > Issue Type: Improvement > Components: Broker, Configuration, JMX, Web Console >Affects Versions: 2.31.0 >Reporter: Gary Tully >Assignee: Gary Tully >Priority: Major > > we have the manage permission that allows sending to the management address, > to access any control resource. We don't however distinguish what a user can > do. > We should segment control operations into categories: CRUD provides a basis > view for get/is (Read) > update for set or operations that mutate or modify. > We allow this sort of configuration via management.xml for jmx mbean access > but using a different model based on object name. > All of the mbeans delegate to the control resources. > If we add these two additional permissions then we can have a single rbac > model (that supports config reload) and more granularity on control resource > access from the management address. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ARTEMIS-4582) add view and update permissions to augment the manage rbac for control resources
[ https://issues.apache.org/jira/browse/ARTEMIS-4582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Tully updated ARTEMIS-4582: Description: we have the manage permission that allows sending to the management address, to access any control resource. We don't however distinguish what a user can do. We should segment control operations into categories: CRUD provides a basis view for get/is (Read) update for set or operations that mutate or modify. We allow this sort of configuration via management.xml for jmx mbean access but using a different model based on object name. All of the mbeans delegate to the control resources. If we add these two additional permissions then we can have a single rbac model (that supports config reload) and more granularity on control resource access from the management address. was: we have the manage permission that allows sending to the management address, to access any control resource. We should segment control operations into categories: CRUD provides a basis view for get/is (Read) edit for set (Update) manage for aggregate operations list* and Create, Delete) also implying both view & edit We allow this sort of configuration via management.xml for jmx mbean access but using a different model based on object name. All of the mbeans delegate to the control resources. If we add these two additional permissions then we can have a single rbac model (that supports config reload) and more granularity on control resource access from the management address. > add view and update permissions to augment the manage rbac for control > resources > > > Key: ARTEMIS-4582 > URL: https://issues.apache.org/jira/browse/ARTEMIS-4582 > Project: ActiveMQ Artemis > Issue Type: Improvement > Components: Broker, Configuration, JMX, Web Console >Affects Versions: 2.31.0 >Reporter: Gary Tully >Priority: Major > > we have the manage permission that allows sending to the management address, > to access any control resource. We don't however distinguish what a user can > do. > We should segment control operations into categories: CRUD provides a basis > view for get/is (Read) > update for set or operations that mutate or modify. > We allow this sort of configuration via management.xml for jmx mbean access > but using a different model based on object name. > All of the mbeans delegate to the control resources. > If we add these two additional permissions then we can have a single rbac > model (that supports config reload) and more granularity on control resource > access from the management address. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (ARTEMIS-1230) Create Maven Bill of Materials (BOM)
[ https://issues.apache.org/jira/browse/ARTEMIS-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robbie Gemmell reopened ARTEMIS-1230: - > Create Maven Bill of Materials (BOM) > > > Key: ARTEMIS-1230 > URL: https://issues.apache.org/jira/browse/ARTEMIS-1230 > Project: ActiveMQ Artemis > Issue Type: Improvement >Reporter: Daniel Siviter >Priority: Major > Time Spent: 7h > Remaining Estimate: 0h > > It's frustrating having to deal with dependency versions so the creation of a > Bill of Materials will assist with this. > I'm currently battling with Wildfly Swarm 2017.6.0 overriding to > v1.1.0.wildfly017. To override the version of Artemis I have to have an entry > in my POM for every single dependency which is arduous. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARTEMIS-1230) Create Maven Bill of Materials (BOM)
[ https://issues.apache.org/jira/browse/ARTEMIS-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robbie Gemmell resolved ARTEMIS-1230. - Fix Version/s: 2.33.0 Resolution: Fixed > Create Maven Bill of Materials (BOM) > > > Key: ARTEMIS-1230 > URL: https://issues.apache.org/jira/browse/ARTEMIS-1230 > Project: ActiveMQ Artemis > Issue Type: Improvement >Reporter: Daniel Siviter >Priority: Major > Fix For: 2.33.0 > > Time Spent: 7h > Remaining Estimate: 0h > > It's frustrating having to deal with dependency versions so the creation of a > Bill of Materials will assist with this. > I'm currently battling with Wildfly Swarm 2017.6.0 overriding to > v1.1.0.wildfly017. To override the version of Artemis I have to have an entry > in my POM for every single dependency which is arduous. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (ARTEMIS-1230) Create Maven Bill of Materials (BOM)
[ https://issues.apache.org/jira/browse/ARTEMIS-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17817323#comment-17817323 ] ASF subversion and git services commented on ARTEMIS-1230: -- Commit 9cd598ebf46dfdd42d8e2b06dd380c471c4c6d47 in activemq-artemis's branch refs/heads/main from Alexey Markevich [ https://gitbox.apache.org/repos/asf?p=activemq-artemis.git;h=9cd598ebf4 ] ARTEMIS-1230 Added artemis-bom > Create Maven Bill of Materials (BOM) > > > Key: ARTEMIS-1230 > URL: https://issues.apache.org/jira/browse/ARTEMIS-1230 > Project: ActiveMQ Artemis > Issue Type: Improvement >Reporter: Daniel Siviter >Priority: Major > Time Spent: 7h > Remaining Estimate: 0h > > It's frustrating having to deal with dependency versions so the creation of a > Bill of Materials will assist with this. > I'm currently battling with Wildfly Swarm 2017.6.0 overriding to > v1.1.0.wildfly017. To override the version of Artemis I have to have an entry > in my POM for every single dependency which is arduous. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (ARTEMIS-1230) Create Maven Bill of Materials (BOM)
[ https://issues.apache.org/jira/browse/ARTEMIS-1230?focusedWorklogId=904847=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-904847 ] ASF GitHub Bot logged work on ARTEMIS-1230: --- Author: ASF GitHub Bot Created on: 14/Feb/24 10:33 Start Date: 14/Feb/24 10:33 Worklog Time Spent: 10m Work Description: gemmellr merged PR #4782: URL: https://github.com/apache/activemq-artemis/pull/4782 Issue Time Tracking --- Worklog Id: (was: 904847) Time Spent: 7h (was: 6h 50m) > Create Maven Bill of Materials (BOM) > > > Key: ARTEMIS-1230 > URL: https://issues.apache.org/jira/browse/ARTEMIS-1230 > Project: ActiveMQ Artemis > Issue Type: Improvement >Reporter: Daniel Siviter >Priority: Major > Time Spent: 7h > Remaining Estimate: 0h > > It's frustrating having to deal with dependency versions so the creation of a > Bill of Materials will assist with this. > I'm currently battling with Wildfly Swarm 2017.6.0 overriding to > v1.1.0.wildfly017. To override the version of Artemis I have to have an entry > in my POM for every single dependency which is arduous. -- This message was sent by Atlassian Jira (v8.20.10#820010)
