Another issue from the estimates was: the broker had the assumption it would update the size at a later point when decoding.. so the broker would lazy initialize the estimates and later update it as needed.
however that later update wouldn't be much more in terms of delta and the message was not well weighted anyways. To correct compute the application properties size, I need to always scan the positions of messages during reload.. so I'm removing that lazyDecode. On Thu, Mar 26, 2026 at 9:50 PM Clebert Suconic <[email protected]> wrote: > > I created this draft pull request, as a discussion point: > > https://github.com/apache/artemis/pull/6323 > > > We used to always estimate application properties, and these are way > lower... even after scanned the update is not bringing the whole size > up. > > > I compared CORE and AMQP on a test, and i decided to Multiply by 4 the > encoding of application properties as a bare estimate. At least now I > can send millions of messages without OME. > > > I will update the PR with a test where I send a few thousand messages > on a broker with limited memory. > > > Also the PR shows a few failures, especially around ElasticQueueTest > because the sizing now has to be updated. > > > > I know this change will be disruptive for users.. but I would rather > have this disrupting a few users, than causing OME's for people using > AMQP producers... it's way better to manage it this way IMO. > > On Tue, Mar 17, 2026 at 2:44 AM Jean-Baptiste Onofré <[email protected]> > wrote: > > > > Hi > > > > It seems that we should re-evaluate the estimate. > > > > Regards > > JB > > > > On Mon, Mar 16, 2026 at 11:02 PM Clebert Suconic > > <[email protected]> wrote: > > > > > > > > > > > Are we currently underestimating the size of AMQP messages? If so, > > > > what exactly was underestimated? > > > > > > > > > If you sent a Core message with ./artemis producer --message-count=1 > > > --protocol=AMQP > > > > > > The address size of that message will be 642 bytes (from the estimate) > > > > > > If you send a Core Message with ./artemis producer --message-count=1 > > > --protocol=CORE > > > > > > The estimate from CORE is using 1121 > > > > > > > > > I believe we need to raise the offset on the AMQP message, raising the > > > minimal. > > > > > > By pegging I mean.. the broker's performance starts to crawl, using a > > > lot of time on GC.. and it may eventually OME. It eventually starts to > > > Page but at that point things are really bad already. > > > > > > > > > > > > There's another change I have beeen wanting to do, that is to keep the > > > estimate static... and I want to do it as part of 2.54.0 as I'm not > > > holding 2.53 for this. > > > > > > > > > > > > > > When you say "pegging" are you talking about CPU utilization? If so, > > > > what is consuming all the CPU? Garbage collection? > > > > > > > > If folks potentially need to adjust their paging config we'll need to > > > > guide them on how to do it in the docs. > > > > > > > > > > > > Justin > > > > > > > > On Mon, Mar 16, 2026 at 2:13 PM Clebert Suconic > > > > <[email protected]> wrote: > > > > > > > > > > As I was "playing" with my changes on Shell, I stumbled on an issue. > > > > > > > > > > > > > > > As I sent millions of messages, Core Messages will trigger paging a > > > > > lot sooner. > > > > > While AMQP Messages will trigger it much later, but the broker starts > > > > > pegging and decreases the performance quite a lot. > > > > > > > > > > > > > > > I played with some adjusting on the estimates, and I will fix it on > > > > > the next release (2.54.. not 2.53).. > > > > > > > > > > Users will have to readjust their estimates though... > > > > > > > > > > What do you think? > > > > > > > > > > -- > > > > > Clebert Suconic > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: [email protected] > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [email protected] > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > -- > > > Clebert Suconic > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > > -- > Clebert Suconic -- Clebert Suconic --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
