> Could not configuring a producer-window-size on the bridge cause the issue noted above? It says that it is disabled by default.
Clearly it's connected at some level, but it's hard to say more without a better understanding of the root-cause. > What is considered a really huge large message? This documentation was added as part of ARTEMIS-94 [1] where the size of a message exceeded the max value of a Java Integer [2] so this appears to be the threshold of "really huge". > Should there not be something in the logs to indicate that something fatal is going here? Possibly. Again, it's hard to say more without understanding the root-cause. > If you do need to configure the producer-window-size, how do you determine the size that it should be set to (especially if you get all sorts of message sizes)? Typically this comes down to tuning performance & providing safety for the remote broker. The higher the producerWindowSize the less often the producer will ask for credits and therefore the less likely it will be to block which means the more likely it is to overwhelm the remote broker. > Is there any reason why we wouldn't just set a very high producer-window-size on all of our bridges? The only time the bridge should not send messages is if the other side is down for some reason. See my previous answer. Can you reproduce this on 2.17.0? How about 2.18.0-SNAPSHOT [3]? Justin [1] https://issues.apache.org/jira/browse/ARTEMIS-94 [2] https://docs.oracle.com/javase/8/docs/api/java/lang/Integer.html#MAX_VALUE [3] https://repository.apache.org/content/repositories/snapshots/org/apache/activemq/apache-artemis/2.18.0-SNAPSHOT/ On Mon, Jun 21, 2021 at 8:14 AM foo bar <statuario...@gmail.com> wrote: > Hello, > > I have managed to replicate an issue that is occuring in our production > environment with the setup noted below. We are using ActiveMQ Artemis > version 2.15. > > There are two brokers (broker1 and broker2). Each broker has only one queue > called test. There is a bridge from broker1 test queue to broker2 test > queue. > > <bridges> > <bridge name="broker1-to-broker2-bridge"> > <queue-name>test</queue-name> > <static-connectors> > <connector-ref>broker2-connector</connector-ref> > </static-connectors> > </bridge> > </bridges> > > Using a java program (JMS API) we create 10 producers. Each producer puts > 200 messages into the broker1 test queue (concurrently). The size of each > message is 70 megabytes.What should happen is that all messages should move > across the bridge and eventually end up in the broker2 test queue. What > happens is approximately 1 message (in some cases more) will make it to the > broker2 test queue, the other messages pile up in the broker1 test queue. > The consumer count on the broker1 test queue is always 1 and the delivering > count is 0. At this point nothing will ever get through the bridge - it is > completely sunk. There are absolutely *no log messages* in broker1 or > broker2 that would indicate something is wrong. Producers can however still > send messages to the broker1 test queue. In our actual production > environment, where all messages are not 70 megabytes, a restart of the > broker will move messages across the bridge until the issue occurs again > and we need to *manually restart the broker*. > > If I rerun the above test with debug on, the only thing that seems to stick > out in the logs is that there are lots of SessionProducerCreditsMessages" > with credits=0 which leads us to the test below. > > If we rerun the our test again with this configuration: > > <bridges> > <bridge name="broker1-to-broker2-bridge"> > <queue-name>test</queue-name> > *<producer-window-size>209715200</producer-window-size>* > <static-connectors> > <connector-ref>broker2-connector</connector-ref> > </static-connectors> > </bridge> > </bridges> > > > The messages all flow from broker1 test queue to broker 2 test queue. The > documentation says this about producer-window-size: > > > - producer-window-size. This optional parameter determines the producer > flow control through the bridge. You usually leave this off unless you > are > dealing with huge large messages > > > *Questions* > > 1.) Could not configuring a producer-window-size on the bridge cause the > issue noted above? It says that it is disabled by default. > 2.) What is considered a really huge large message? > 3.) Should there not be something in the logs to indicate that something > fatal is going here? > 4.) If you do need to configure the producer-window-size, how do you > determine the size that it should be set to (especially if you get all > sorts of message sizes)? > 5.) Is there any reason why we wouldn't just set a very high > producer-window-size on all of our bridges? The only time the bridge should > not send messages is if the other side is down for some reason. > > The issue is with Artemis 2.15, but we can also reproduce this on 2.16. > > Thanks >
