[jira] [Commented] (ROCKETMQ-253) Compress RegisterBrokerBody
[ https://issues.apache.org/jira/browse/ROCKETMQ-253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119380#comment-16119380 ] ASF GitHub Bot commented on ROCKETMQ-253: - Github user shroman commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/140#discussion_r132086276 --- Diff: broker/src/main/java/org/apache/rocketmq/broker/out/BrokerOuterAPI.java --- @@ -148,12 +148,13 @@ private RegisterBrokerResult registerBroker( requestHeader.setBrokerName(brokerName); requestHeader.setClusterName(clusterName); requestHeader.setHaServerAddr(haServerAddr); +requestHeader.setCompressed(true); RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.REGISTER_BROKER, requestHeader); RegisterBrokerBody requestBody = new RegisterBrokerBody(); requestBody.setTopicConfigSerializeWrapper(topicConfigWrapper); requestBody.setFilterServerList(filterServerList); -request.setBody(requestBody.encode()); +request.setBody(requestBody.encode(true)); --- End diff -- Great idea! How about setting a threshold for compression being triggered, because if the data you compress is small, it overhead is higher than the benefits you get from the compression? > Compress RegisterBrokerBody > --- > > Key: ROCKETMQ-253 > URL: https://issues.apache.org/jira/browse/ROCKETMQ-253 > Project: Apache RocketMQ > Issue Type: Improvement > Components: rocketmq-broker, rocketmq-commons, rocketmq-namesrv >Reporter: Zhanhui Li >Assignee: yukon > > RocketMQ Brokers periodically register itself against namesrv to maintain its > liveness. > In case a broker has many topics, a 10 thousand or more, two major > problematic issues show up: > 1) Current serialization of RegisterBrokerBody would create a large String > object, which stresses GC a lot, especially when G1 is used. > 2) Transmission this bulk of data takes up significant portion of bandwidth. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ROCKETMQ-242) mqclient can not fetch nameSrvAddr periodly
[
https://issues.apache.org/jira/browse/ROCKETMQ-242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118312#comment-16118312
]
ASF GitHub Bot commented on ROCKETMQ-242:
-
Github user qqeasonchen closed the pull request at:
https://github.com/apache/incubator-rocketmq/pull/139
> mqclient can not fetch nameSrvAddr periodly
> ---
>
> Key: ROCKETMQ-242
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-242
> Project: Apache RocketMQ
> Issue Type: Bug
> Components: rocketmq-client
>Affects Versions: 4.0.0-incubating, 4.1.0-incubating
> Environment: test and production
>Reporter: Eason Chen
>Assignee: Xiaorui Wang
> Fix For: 4.2.0-incubating
>
>
> when mqclientinstance start, it getNameSrvAddr once by bellow code:
> if (null == this.clientConfig.getNamesrvAddr()) {
>
> this.clientConfig.setNamesrvAddr(this.mQClientAPIImpl.fetchNameServerAddr());
> }
> but in startScheduledTask()，it will not start the schedule task if
> namesrvAddr is not null
> code=
> {code:java}
> public void start() throws MQClientException {
> synchronized (this) {
> switch (this.serviceState) {
> case CREATE_JUST:
> this.serviceState = ServiceState.START_FAILED;
> // If not specified,looking address from name server
>*{color:red} if (null ==
> this.clientConfig.getNamesrvAddr()) {
>
> this.clientConfig.setNamesrvAddr(this.mQClientAPIImpl.fetchNameServerAddr());
> }{color}*
> // Start request-response channel
> this.mQClientAPIImpl.start();
> // Start various schedule tasks
> this.startScheduledTask();
> // Start pull service
> this.pullMessageService.start();
> // Start rebalance service
> this.rebalanceService.start();
> // Start push service
>
> this.defaultMQProducer.getDefaultMQProducerImpl().start(false);
> log.info("the client factory [{}] start OK",
> this.clientId);
> this.serviceState = ServiceState.RUNNING;
> break;
> case RUNNING:
> break;
> case SHUTDOWN_ALREADY:
> break;
> case START_FAILED:
> throw new MQClientException("The Factory object[" +
> this.getClientId() + "] has been created before, and failed.", null);
> default:
> break;
> }
> }
> }
> private void startScheduledTask() {
> *{color:red} if (null == this.clientConfig.getNamesrvAddr()) {{color}*
> this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
> @Override
> public void run() {
> try {
>
> MQClientInstance.this.mQClientAPIImpl.fetchNameServerAddr();
> } catch (Exception e) {
> log.error("ScheduledTask fetchNameServerAddr
> exception", e);
> }
> }
> }, 1000 * 10, 1000 * 60 * 2, TimeUnit.MILLISECONDS);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ROCKETMQ-242) mqclient can not fetch nameSrvAddr periodly
[
https://issues.apache.org/jira/browse/ROCKETMQ-242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118059#comment-16118059
]
ASF GitHub Bot commented on ROCKETMQ-242:
-
Github user Jaskey commented on a diff in the pull request:
https://github.com/apache/incubator-rocketmq/pull/139#discussion_r131043380
--- Diff:
client/src/main/java/org/apache/rocketmq/client/impl/factory/MQClientInstance.java
---
@@ -255,19 +255,17 @@ public void start() throws MQClientException {
}
private void startScheduledTask() {
-if (null == this.clientConfig.getNamesrvAddr()) {
-this.scheduledExecutorService.scheduleAtFixedRate(new
Runnable() {
+this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
--- End diff --
I don't see the problem if you agree that: dynamical fetch should only be
trigged when user does not specify the ns explictly.
So it means
if(USER DOES NOT SPECIFY NS) {
startFetch
}
where USER DOES NOT SPECIFY NS equals to `null ==
this.clientConfig.getNamesrvAddr()`
> mqclient can not fetch nameSrvAddr periodly
> ---
>
> Key: ROCKETMQ-242
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-242
> Project: Apache RocketMQ
> Issue Type: Bug
> Components: rocketmq-client
>Affects Versions: 4.0.0-incubating, 4.1.0-incubating
> Environment: test and production
>Reporter: Eason Chen
>Assignee: Xiaorui Wang
> Fix For: 4.2.0-incubating
>
>
> when mqclientinstance start, it getNameSrvAddr once by bellow code:
> if (null == this.clientConfig.getNamesrvAddr()) {
>
> this.clientConfig.setNamesrvAddr(this.mQClientAPIImpl.fetchNameServerAddr());
> }
> but in startScheduledTask()，it will not start the schedule task if
> namesrvAddr is not null
> code=
> {code:java}
> public void start() throws MQClientException {
> synchronized (this) {
> switch (this.serviceState) {
> case CREATE_JUST:
> this.serviceState = ServiceState.START_FAILED;
> // If not specified,looking address from name server
>*{color:red} if (null ==
> this.clientConfig.getNamesrvAddr()) {
>
> this.clientConfig.setNamesrvAddr(this.mQClientAPIImpl.fetchNameServerAddr());
> }{color}*
> // Start request-response channel
> this.mQClientAPIImpl.start();
> // Start various schedule tasks
> this.startScheduledTask();
> // Start pull service
> this.pullMessageService.start();
> // Start rebalance service
> this.rebalanceService.start();
> // Start push service
>
> this.defaultMQProducer.getDefaultMQProducerImpl().start(false);
> log.info("the client factory [{}] start OK",
> this.clientId);
> this.serviceState = ServiceState.RUNNING;
> break;
> case RUNNING:
> break;
> case SHUTDOWN_ALREADY:
> break;
> case START_FAILED:
> throw new MQClientException("The Factory object[" +
> this.getClientId() + "] has been created before, and failed.", null);
> default:
> break;
> }
> }
> }
> private void startScheduledTask() {
> *{color:red} if (null == this.clientConfig.getNamesrvAddr()) {{color}*
> this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
> @Override
> public void run() {
> try {
>
> MQClientInstance.this.mQClientAPIImpl.fetchNameServerAddr();
> } catch (Exception e) {
> log.error("ScheduledTask fetchNameServerAddr
> exception", e);
> }
> }
> }, 1000 * 10, 1000 * 60 * 2, TimeUnit.MILLISECONDS);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
