[
https://issues.apache.org/activemq/browse/AMQ-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fabrice Delaporte updated AMQ-2532:
-----------------------------------
Description:
There seems to be a regression (works fine in 5.2.0) related to the fix for the
following:
https://issues.apache.org/activemq/browse/AMQ-2413
I think the intent of the original code was to prevent uncontrolled paging of
pending messages. The doPageIn method is called in version 5.3.0 :
- each time a message is acked/committed
- each time the MemoryUsage is decreased
As a result, if you create a queue with, say, 1 000 000 messages in it, then
start a consumer, the broker will try to page in all the messages from the
store as consuming goes on (with default configuration values, 200 paged in
messages for one consumed message minimum - I say minimum because if the
MemoryUsage is decreased when you consume a message, then doPageIn is invoked
again). Guaranteed OutOfMemoryError as in general consuming messages is slower
than paging from store. I guess the same can occur when you have a fast
producer and a slow consumer, without producer flow control.
I would say that in any case, the broker should never page in more messages
than the maxPageSize attribute, plus the number of dispatched messages. More
formally:
{code:java}
int toPageIn = getMaxPageSize() - pagedInPendingDispatch.size();
{code}
And just to make sure we never get a negative value:
{code:java}
int toPageIn = Math.max(0, getMaxPageSize() -
pagedInPendingDispatch.size());
{code}
Note 1: this behavior is also exhibited by the background message expiration
logic, that forces message pagination each time it is run. With big queues
without consumers, by default you'll get 200 more messages paginated each time
the expiration task is run. Eventually leading to broker resource exhaustion if
no consumer comes into play in the near future with active producers queuing up
messages.
Note 2: this may also be related to
https://issues.apache.org/activemq/browse/AMQ-2468
Workarounds:
- setting lazyDispatch to true fixes the problem for message consuming only
- background expiration can be disabled by setting expireMessagesPeriod to -1
Cheers,
Fabrice
was:
There seems to be a regression (works fine in 5.2.0) related to the fix for the
following:
https://issues.apache.org/activemq/browse/AMQ-2413
I think the intent of the original code was to prevent uncontrolled paging of
pending messages. The doPageIn method is called in version 5.3.0 :
- each time a message is acked/committed
- each time the MemoryUsage is decreased
As a result, if you create a queue with, say, 1 000 000 messages in it, then
start a consumer, the broker will try to page in all the messages from the
store as consuming goes on (with default configuration values, 200 paged in
messages for one consumed message minimum - I say minimum because if the
MemoryUsage is decreased when you consume a message, then doPageIn is invoked
again). Guaranteed OutOfMemoryError as in general consuming messages is slower
than paging from store. I guess the same can occur when you have a fast
producer and a slow consumer, without producer flow control.
I would say that in any case, the broker should never page in more messages
than the maxPageSize attribute, plus the number of dispatched messages. More
formally:
{code:java}
int toPageIn = getMaxPageSize() - pagedInPendingDispatch.size();
{code}
And just to make sure we never get a negative value:
{code:java}
int toPageIn = Max(0, getMaxPageSize() -
pagedInPendingDispatch.size());
{code}
Note 1: this behavior is also exhibited by the background message expiration
logic, that forces message pagination each time it is run. With big queues
without consumers, by default you'll get 200 more messages paginated each time
the expiration task is run. Eventually leading to broker resource exhaustion if
no consumer comes into play in the near future with active producers queuing up
messages.
Note 2: this may also be related to
https://issues.apache.org/activemq/browse/AMQ-2468
Workarounds:
- setting lazyDispatch to true fixes the problem for message consuming only
- background expiration can be disabled by setting expireMessagesPeriod to -1
Cheers,
Fabrice
> Broker paging all available messages from store leading to resource exhaustion
> ------------------------------------------------------------------------------
>
> Key: AMQ-2532
> URL: https://issues.apache.org/activemq/browse/AMQ-2532
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: Java 1.6
> Reporter: Fabrice Delaporte
>
> There seems to be a regression (works fine in 5.2.0) related to the fix for
> the following:
> https://issues.apache.org/activemq/browse/AMQ-2413
> I think the intent of the original code was to prevent uncontrolled paging of
> pending messages. The doPageIn method is called in version 5.3.0 :
> - each time a message is acked/committed
> - each time the MemoryUsage is decreased
> As a result, if you create a queue with, say, 1 000 000 messages in it, then
> start a consumer, the broker will try to page in all the messages from the
> store as consuming goes on (with default configuration values, 200 paged in
> messages for one consumed message minimum - I say minimum because if the
> MemoryUsage is decreased when you consume a message, then doPageIn is invoked
> again). Guaranteed OutOfMemoryError as in general consuming messages is
> slower than paging from store. I guess the same can occur when you have a
> fast producer and a slow consumer, without producer flow control.
> I would say that in any case, the broker should never page in more messages
> than the maxPageSize attribute, plus the number of dispatched messages. More
> formally:
> {code:java}
> int toPageIn = getMaxPageSize() - pagedInPendingDispatch.size();
> {code}
> And just to make sure we never get a negative value:
> {code:java}
> int toPageIn = Math.max(0, getMaxPageSize() -
> pagedInPendingDispatch.size());
> {code}
> Note 1: this behavior is also exhibited by the background message expiration
> logic, that forces message pagination each time it is run. With big queues
> without consumers, by default you'll get 200 more messages paginated each
> time the expiration task is run. Eventually leading to broker resource
> exhaustion if no consumer comes into play in the near future with active
> producers queuing up messages.
> Note 2: this may also be related to
> https://issues.apache.org/activemq/browse/AMQ-2468
> Workarounds:
> - setting lazyDispatch to true fixes the problem for message consuming only
> - background expiration can be disabled by setting expireMessagesPeriod to -1
> Cheers,
> Fabrice
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
