I was just wondering if there was a way to disable, or at least force globally
from server side, a value for that parameter.
I mean, the default value is in some way enforced by the broker if no override
is done. So, in the same way, I was wondering if this default parameter could
be changed without adding any parameter to acceptors or connection strings
(according to the journal buffer size, obviously, that anyway could be changed
by configuration right? - <journal-buffer-size>my_value</journal-buffer-size>)
I mentioned MQTT clients to give a picture of our configuration.
Our Artemis exposes 3 acceptors. 2 MQTT for IoT clients with 2 ways load (each
client publishes and subscribes different topics) and one AMQP for consumers
(Camel + streams configured) implementing business logic.
The Camel routes steps have generic signature (Exchange, Object) and in the
full stack trace (I partially reported here) there wasn't any of our class
involved. (I know Object is redundant, but I preferred instead of getting it
from exchange.in in every method)
With my original route configuration, I tried to set the amqpMinLargeMessage
and minLargeMessage to both the acceptor and the connection string on client
side but without any effect.
I already received support from Camel mailing list and they suggested me to
switch to Camel-jms since Camel-amqp doesn’t support the minLargeMessage
(though in my tests I was able to get the stream from Artemis once I switched
to use core instead of amqp)
I’m looking forward to see the minLargeMessage supported by Camel streams also
with amqp endpoint.
From: Robbie Gemmell <robbie.gemm...@gmail.com>
Date: Monday, 4 September 2023 at 14:11
To: users@activemq.apache.org <users@activemq.apache.org>
Subject: Re: Disable large messages support at all
Though actually, another thing occurred to me...you said your "clients
are MQTT", in which case the Artemis client minLargeMessageSize config
and broker side amqpMinLargeMessageSize config wont necessarily be in
play if your Camel bits are only consuming messages, as those configs
primarily affect what happens when the Core and AMQP clients are
producing messages. Its not really clear so far what all you are doing
at each component.
On Mon, 4 Sept 2023 at 12:34, Robbie Gemmell <robbie.gemm...@gmail.com> wrote:
>
> The Artemis broker / client handling of 'large' messages, and
> 'streaming payload to/from file at client' are 2 separate things and
> shouldnt be conflated.
>
> The Artemis 'Core' JMS client + broker consider messages over 100kb by
> default 'large' and treat them a little differently in terms of how
> they are sent by the producer client, and then the broker storing them
> differently on disk, and sending them to consuming clients.
>
> The Artemis client can separately leverage a custom Object property to
> read/write the BytesMessage body using an InputStream / OutputStream,
> e.g to from/to a file, at the application side.
>
> The exception below seems (not 100% clear, as the stack isnt present)
> to be saying that while trying to extract the body of a message Camel
> actually tried to write to a read-only JMS Message (specifically the
> properties of it), and so the qpid-jms client threw because of that,
> as it should. That seems like potentially either an issue with your
> configuration or a bug in Camel. Guessing, is it possible you/Camel
> perhaps configuring things to try inserting the custom 'input/output
> stream' Object property used by the Artemis JMS client, even though it
> isnt actually using the Artemis client? That certainly might explain
> the seeming 'set property while trying to extract body from read-only
> message' behaviour. As you are asking here to 'disable large messages'
> entirely, it sounds like your messages aren't necessarily all that big
> and you dont necessarily need/want the 'stream message to/from file'
> input/output stream behaviour at the client side anyway, and if so you
> can disable that?
>
> I see you mentioned Camel 3.11. That's pretty old now, perhaps also
> try a more recent version to see if it fixed anything that might be in
> play here?
>
> There is a second broker-side acceptor URL config parameter,
> amqpMinLargeMessageSize, that defaults to 102400 (100kb) and can
> change the point at which the broker considers AMQP messages to be
> 'large' and handles them on disk in similar way as it does the Artemis
> Core 'large' messages. Though there are also other considerations
> though as Justin said, such as the journal buffer size and file size,
> so its not a case of just setting it really large to disable the
> behaviour entirely.
>
> On Mon, 4 Sept 2023 at 08:32, Modanese, Riccardo
> <riccardo.modan...@eurotech.com.invalid> wrote:
> >
> > Hi Justin,
> > Thank you for your reply.
> > I had no way to get large messages support using Camel routes with
> > Camel-amqp endpoint.
> > With Camel-amqp, using qpid connection
> > (org.apache.qpid.jms.JmsConnectionFactory), messages over the default 100kb
> > weren’t processed.
> > Camel went to an infinite loop printing this error at each iteration:
> >
> > 2023-08-23 11:07:18 org.apache.camel.RuntimeCamelException: Failed to
> > extract body due to: javax.jms.MessageNotWriteableException: Message
> > properties are read-only. Message: JmsBytesMessage {
> > org.apache.qpid.jms.provider.amqp.message.AmqpJmsBytesMessageFacade@39dd152c<mailto:org.apache.qpid.jms.provider.amqp.message.AmqpJmsBytesMessageFacade@39dd152c>
> > }
> > 2023-08-23 11:07:18 at
> > org.apache.camel.component.jms.JmsBinding.extractBodyFromJms(JmsBinding.java:174)
> > 2023-08-23 11:07:18 at
> > org.apache.camel.component.jms.JmsMessage.createBody(JmsMessage.java:234)
> >
> > Switching to core the messages over 100k were handled but no way to change
> > the minimum large messages default value.
> >
> > From the support I got from Camel mailing list there is no other way than
> > using Camel-jms to have full large messages support:
> >
> > “I am afraid that in order to use minLargeMessageSize you need to use
> > *camel-jms* component and *artemis-jms-client* as jms implementation,
> > because camel-amqp uses *qpid-jms-client.*If you have some time, you can
> > take a look to camel-jms”
> >
> > This force us to use the Artemis connection factory
> > (org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory), in our
> > clients, to connect to the broker, decreasing the generality we like to
> > have on our opensource project.
> >
> > I’m still testing a working configuration with Camel-jms but I was just
> > wondering if there was a way to disable the large messages support.
> >
> > Riccardo
> >
> > From: Justin Bertram <jbert...@apache.org>
> > Date: Friday, 1 September 2023 at 20:56
> > To: users@activemq.apache.org <users@activemq.apache.org>
> > Subject: Re: Disable large messages support at all
> > There is no way to completely disable large message support. This is
> > because clients are free to send messages larger than the configured
> > journal-buffer-size (490KB by default). When that happens the broker has no
> > choice but to treat the message as "large" and stream it to disk rather
> > than storing it in the journal. You can increase the journal-buffer-size to
> > make large messages less likely or even effectively eliminate them
> > depending on your use-case, but using the larger buffer may have an impact
> > on performance.
> >
> > What "difficulties" are you having currently with AMQP, MQTT, & large
> > messages?
> >
> >
> > Justin
> >
> >
> > On Wed, Aug 30, 2023 at 2:21 AM Modanese, Riccardo
> > <riccardo.modan...@eurotech.com.invalid> wrote:
> >
> > > Hello to everyone,
> > >
> > > I’m trying to configure my Camel routes (3.11) with Artemis (2.28.0)
> > > large messages but I’m still experiencing some difficulties.
> > > My clients are MQTT and my Camel routes are using AMQP endpoint.
> > > Is there a way to completely disable large message support on Artemis or
> > > to set the min value globally on Artemis to override any setting made on
> > > connectors bases?
> > >
> > > Regards,
> > >
> > > Riccardo Modanese
> > >
> > >
