[
https://issues.apache.org/jira/browse/ROCKETMQ-294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16178940#comment-16178940
]
ASF GitHub Bot commented on ROCKETMQ-294:
-----------------------------------------
GitHub user zhouxinyu opened a pull request:
https://github.com/apache/incubator-rocketmq/pull/171
[ROCKETMQ-294] Do flow control on the number and size dimensions when pull
message
## What is the purpose of the change
Current flow control strategy only support on Queue level, each message
queue can cache 1000 message by default.
When lots of message queue are assigned to one consumer instance, the
consumer will cache too many messages, may cause OOM exception.
On the other hand, only control the message amount is not enough, should
support do flow control on the number and size dimensions, on Topic and Queue
level.
## Brief changelog
Do flow control on the number and size dimensions when pulling message
## Verifying this change
This PR added three parameters, plus the old one, there are 4 parameters
for flow control:
* pullThresholdForQueue, flow control threshold on queue level, each
message queue will cache at most 1000 messages by default.
* pullThresholdForTopic, flow control threshold on topic level, default
value is -1(Unlimited)
* pullThresholdSizeForQueue, limit the cached message size on queue level,
each message queue will cache at most 100 MiB messages by default.
* pullThresholdSizeForTopic, limit the cached message size on topic level,
the default value is -1 MiB(Unlimited).
See the javadoc for details, you can change the value, and observe the log
to ensure it works.
Follow this checklist to help us incorporate your contribution quickly and
easily:
- [x] Make sure there is a [JIRA
issue](https://issues.apache.org/jira/browse/ROCKETMQ-294) filed for the change
(usually before you start working on it). Trivial changes like typos do not
require a JIRA issue. Your pull request should address just this issue, without
pulling in other changes - one PR resolves one issue.
- [x] Format the pull request title like `[ROCKETMQ-XXX] Fix
UnknownException when host config not exist`. Each commit in the pull request
should have a meaningful subject line and body.
- [x] Write a pull request description that is detailed enough to
understand what the pull request does, how, and why.
- [x] Write necessary unit-test to verify your logic correction, more mock
a little better when cross module dependency exist. If the new feature or
significant change is committed, please remember to add integration-test in
[test module](https://github.com/apache/incubator-rocketmq/tree/master/test).
- [x] Run `mvn -B clean apache-rat:check findbugs:findbugs
checkstyle:checkstyle` to make sure basic checks pass. Run `mvn clean install
-DskipITs` to make sure unit-test pass. Run `mvn clean test-compile
failsafe:integration-test` to make sure integration-test pass.
- [x] If this contribution is large, please file an [Apache Individual
Contributor License Agreement](http://www.apache.org/licenses/#clas).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zhouxinyu/incubator-rocketmq ROCKETMQ-294
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-rocketmq/pull/171.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #171
----
commit 8878a21e2d0a87ac0d1ae59483eabcc34fdafb07
Author: yukon <[email protected]>
Date: 2017-09-25T07:55:55Z
[ROCKETMQ-294] Do flow control on the number and size dimensions when pull
message
----
> Do flow control on the number and size dimensions when pull message
> -------------------------------------------------------------------
>
> Key: ROCKETMQ-294
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-294
> Project: Apache RocketMQ
> Issue Type: Improvement
> Components: rocketmq-client
> Reporter: yukon
> Assignee: yukon
> Fix For: 4.2.0-incubating
>
>
> *Motivations*
> Current flow control strategy only support on Queue level, each message queue
> can cache 1000 message by default.
> When lots of message queue are assigned to one consumer instance, the
> consumer will cache too many messages, may cause OOM exception.
> On the other hand, only control the message amount is not enough, should
> support do flow control on the number and size dimensions, on Topic and Queue
> level.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
