[jira] [Comment Edited] (ROCKETMQ-169) Support flow control when slave restarts and replicating from master

2017-04-12 Thread Jaskey Lam (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965784#comment-15965784
 ] 

Jaskey Lam edited comment on ROCKETMQ-169 at 4/12/17 12:57 PM:
---

I think this is an important and beneficial feature, since if a new slave or 
slave who is falling behind much starts, the out bandwidth will be occupied 
much and this will influence the consuming tps and synchronization of another 
healthy slave.

A trafic flow control should be done.

In my opinion, we can add the traffic control code logic in the master when 
process reading data from slave for:
{%code%}
processWriteEvent()...//existing trasfer logic
...
if (isSlaveRequestAVerySmallOffset) {
 computeTpsAndDoFlowControl();//sleep for while 
} 

   {%code%}

[~Yukon]
[~lizhanhui]
[~vongosling]

what's your advice?


was (Author: jaskey):
I think this is an important and beneficial feature, since if a new slave or 
slave who is falling behind much starts, the out bandwidth will be occupied 
much and this will influence the consuming tps and synchronization of another 
healthy slave.

A trafic flow control should be done.

In my opinion, we can add the traffic control code logic in the master when 
process reading data from slave for:

processWriteEvent()...
...
if (isSlaveRequestAVerySmallOffset) {
 computeTpsAndDoFlowControl();//sleep for while 
} 


[~Yukon]
[~lizhanhui]
[~vongosling]

what's your advice?

> Support flow control when slave restarts and replicating from master
> 
>
> Key: ROCKETMQ-169
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-169
> Project: Apache RocketMQ
>  Issue Type: Improvement
>  Components: rocketmq-broker
>Affects Versions: 4.0.0-incubating
>Reporter: Eason Chen
>Assignee: yukon
>
> as we know slave replicates message from master, but when slave restarts the 
> replicating will cause huge traffic which will impact sending messages, so we 
> suggest supporting flow control for slave when restarts.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-169) Support flow control when slave restarts and replicating from master

2017-04-12 Thread Jaskey Lam (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965784#comment-15965784
 ] 

Jaskey Lam commented on ROCKETMQ-169:
-

I think this is an important and beneficial feature, since if a new slave or 
slave who is falling behind much starts, the out bandwidth will be occupied 
much and this will influence the consuming tps and synchronization of another 
healthy slave.

A trafic flow control should be done.

In my opinion, we can add the traffic control code logic in the master when 
process reading data from slave for:

processWriteEvent()...
...
if (isSlaveRequestAVerySmallOffset) {
 computeTpsAndDoFlowControl();//sleep for while 
} 


[~Yukon]
[~lizhanhui]
[~vongosling]

what's your advice?

> Support flow control when slave restarts and replicating from master
> 
>
> Key: ROCKETMQ-169
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-169
> Project: Apache RocketMQ
>  Issue Type: Improvement
>  Components: rocketmq-broker
>Affects Versions: 4.0.0-incubating
>Reporter: Eason Chen
>Assignee: yukon
>
> as we know slave replicates message from master, but when slave restarts the 
> replicating will cause huge traffic which will impact sending messages, so we 
> suggest supporting flow control for slave when restarts.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (ROCKETMQ-169) Support flow control when slave restarts and replicating from master

2017-04-12 Thread Jaskey Lam (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965784#comment-15965784
 ] 

Jaskey Lam edited comment on ROCKETMQ-169 at 4/12/17 12:58 PM:
---

I think this is an important and beneficial feature, since if a new slave or 
slave who is falling behind much starts, the out bandwidth will be occupied 
much and this will influence the consuming tps and synchronization of another 
healthy slave.

A trafic flow control should be done.

In my opinion, we can add the traffic control code logic in the master when 
process reading data from slave for:

{%code%}

processWriteEvent()...//existing trasfer logic
...
if (isSlaveRequestAVerySmallOffset) {
 computeTpsAndDoFlowControl();//sleep for while 
} 

{%code%}

[~Yukon]
[~lizhanhui]
[~vongosling]

what's your advice?


was (Author: jaskey):
I think this is an important and beneficial feature, since if a new slave or 
slave who is falling behind much starts, the out bandwidth will be occupied 
much and this will influence the consuming tps and synchronization of another 
healthy slave.

A trafic flow control should be done.

In my opinion, we can add the traffic control code logic in the master when 
process reading data from slave for:
{%code%}
processWriteEvent()...//existing trasfer logic
...
if (isSlaveRequestAVerySmallOffset) {
 computeTpsAndDoFlowControl();//sleep for while 
} 

   {%code%}

[~Yukon]
[~lizhanhui]
[~vongosling]

what's your advice?

> Support flow control when slave restarts and replicating from master
> 
>
> Key: ROCKETMQ-169
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-169
> Project: Apache RocketMQ
>  Issue Type: Improvement
>  Components: rocketmq-broker
>Affects Versions: 4.0.0-incubating
>Reporter: Eason Chen
>Assignee: yukon
>
> as we know slave replicates message from master, but when slave restarts the 
> replicating will cause huge traffic which will impact sending messages, so we 
> suggest supporting flow control for slave when restarts.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (ROCKETMQ-169) Support flow control when slave restarts and replicating from master

2017-04-12 Thread Jaskey Lam (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965784#comment-15965784
 ] 

Jaskey Lam edited comment on ROCKETMQ-169 at 4/12/17 12:59 PM:
---

I think this is an important and beneficial feature, since if a new slave or 
slave who is falling behind much starts, the out bandwidth will be occupied 
much and this will influence the consuming tps and synchronization of another 
healthy slave.

A trafic flow control should be done.

In my opinion, we can add the traffic control code logic in the master when 
process reading data from slave for:

{code}

processWriteEvent()...//existing trasfer logic
...
if (isSlaveRequestAVerySmallOffset) {
 computeTpsAndDoFlowControl();//sleep for while 
} 

{code}

[~Yukon]
[~lizhanhui]
[~vongosling]

what's your advice?


was (Author: jaskey):
I think this is an important and beneficial feature, since if a new slave or 
slave who is falling behind much starts, the out bandwidth will be occupied 
much and this will influence the consuming tps and synchronization of another 
healthy slave.

A trafic flow control should be done.

In my opinion, we can add the traffic control code logic in the master when 
process reading data from slave for:

{%code%}

processWriteEvent()...//existing trasfer logic
...
if (isSlaveRequestAVerySmallOffset) {
 computeTpsAndDoFlowControl();//sleep for while 
} 

{%code%}

[~Yukon]
[~lizhanhui]
[~vongosling]

what's your advice?

> Support flow control when slave restarts and replicating from master
> 
>
> Key: ROCKETMQ-169
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-169
> Project: Apache RocketMQ
>  Issue Type: Improvement
>  Components: rocketmq-broker
>Affects Versions: 4.0.0-incubating
>Reporter: Eason Chen
>Assignee: yukon
>
> as we know slave replicates message from master, but when slave restarts the 
> replicating will cause huge traffic which will impact sending messages, so we 
> suggest supporting flow control for slave when restarts.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-166) onException callback may capture compressed message body

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965959#comment-15965959
 ] 

ASF GitHub Bot commented on ROCKETMQ-166:
-

Github user lizhanhui commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/89
  
Comment to bump up this PR.


> onException callback may capture compressed message body
> 
>
> Key: ROCKETMQ-166
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-166
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-client
>Affects Versions: 4.0.0-incubating, 4.1.0-incubating
>Reporter: Zhanhui Li
>Assignee: Zhanhui Li
> Fix For: 4.1.0-incubating
>
>
> If message body size exceeds specified threshold, client would try to 
> compress the message body. 
> Here there are two issues: 1) current implementation changes message body 
> directly, which is not a good practice; 2) if asynchronous send method is 
> employed to deliver message, when onException is invoked, the callback may 
> capture the compressed message body before the finally block restores it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-158) Remove logback dependency for rocketmq-tools

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965968#comment-15965968
 ] 

ASF GitHub Bot commented on ROCKETMQ-158:
-

Github user lizhanhui commented on a diff in the pull request:

https://github.com/apache/incubator-rocketmq/pull/85#discussion_r67737
  
--- Diff: 
common/src/main/java/org/apache/rocketmq/common/utils/LogUtils.java ---
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.common.utils;
+
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.net.URL;
+import org.slf4j.ILoggerFactory;
+import org.slf4j.LoggerFactory;
+
+public class LogUtils {
+
+/**
+ * config logback dymatically from provided file path, otherwise try 
to config from resource file
--- End diff --

Typo here: dymatically --> dynamically


> Remove logback dependency for rocketmq-tools
> 
>
> Key: ROCKETMQ-158
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-158
> Project: Apache RocketMQ
>  Issue Type: Improvement
>  Components: rocketmq-tools
>Reporter: Jaskey Lam
>Assignee: Jaskey Lam
>Priority: Minor
>
> Since user may need to use some admin interfaces to maintain something like 
> create topic, manage queues.
> They will need to use rocketmq-tools which contains DefaultMQAdminExt.
> But rocketmq-tools has explicitly depend on logback-classic and logback-core, 
> which may be conflict with the logging framework of the user's application.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-158) Remove logback dependency for rocketmq-tools

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965972#comment-15965972
 ] 

ASF GitHub Bot commented on ROCKETMQ-158:
-

Github user lizhanhui commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/85
  
+1


> Remove logback dependency for rocketmq-tools
> 
>
> Key: ROCKETMQ-158
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-158
> Project: Apache RocketMQ
>  Issue Type: Improvement
>  Components: rocketmq-tools
>Reporter: Jaskey Lam
>Assignee: Jaskey Lam
>Priority: Minor
>
> Since user may need to use some admin interfaces to maintain something like 
> create topic, manage queues.
> They will need to use rocketmq-tools which contains DefaultMQAdminExt.
> But rocketmq-tools has explicitly depend on logback-classic and logback-core, 
> which may be conflict with the logging framework of the user's application.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (ROCKETMQ-173) Add comment to HA code base

2017-04-12 Thread Jaskey Lam (JIRA) 
Jaskey Lam created ROCKETMQ-173:
---

 Summary: Add comment to HA code base
 Key: ROCKETMQ-173
 URL: https://issues.apache.org/jira/browse/ROCKETMQ-173
 Project: Apache RocketMQ
  Issue Type: Improvement
  Components: rocketmq-store
Reporter: Jaskey Lam
Assignee: yukon


The HA service is the very crucial part for rocketmq and the code base is hard 
to understand.

For now, it is lack of comment in the code base which makes it even harder.

The comment should be added in the code base to make it more easy to understand 
and maintain.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-165) Maximum pull batch size hard-coded restricted

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965902#comment-15965902
 ] 

ASF GitHub Bot commented on ROCKETMQ-165:
-

Github user lizhanhui commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/88
  
Merged as this is really trivial.


> Maximum pull batch size hard-coded restricted
> -
>
> Key: ROCKETMQ-165
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-165
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-store
>Affects Versions: 4.1.0-incubating
>Reporter: Zhanhui Li
>Assignee: Zhanhui Li
>Priority: Minor
> Fix For: 4.1.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-89) WS_DOMAIN_NAME, SUBGROUP default values overrides custom values passed by java options

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965956#comment-15965956
 ] 

ASF GitHub Bot commented on ROCKETMQ-89:


Github user lizhanhui closed the pull request at:

https://github.com/apache/incubator-rocketmq/pull/55


> WS_DOMAIN_NAME, SUBGROUP default values overrides custom values passed by 
> java options
> --
>
> Key: ROCKETMQ-89
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-89
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-broker
>Reporter: Zhanhui Li
>Assignee: Zhanhui Li
>Priority: Minor
> Fix For: 4.1.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-89) WS_DOMAIN_NAME, SUBGROUP default values overrides custom values passed by java options

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965955#comment-15965955
 ] 

ASF GitHub Bot commented on ROCKETMQ-89:


Github user lizhanhui commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/55
  
Merged.


> WS_DOMAIN_NAME, SUBGROUP default values overrides custom values passed by 
> java options
> --
>
> Key: ROCKETMQ-89
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-89
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-broker
>Reporter: Zhanhui Li
>Assignee: Zhanhui Li
>Priority: Minor
> Fix For: 4.1.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ROCKETMQ-173) Add comment to HA code base

2017-04-12 Thread Jaskey Lam (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ROCKETMQ-173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jaskey Lam updated ROCKETMQ-173:

Description: 
The HA service is the very crucial part for rocketmq and the code base is hard 
to understand.

For now, it is lack of comment in the code base which makes it even harder.

The comment should be added in the code base to make it  easier to understand 
and maintain.

  was:
The HA service is the very crucial part for rocketmq and the code base is hard 
to understand.

For now, it is lack of comment in the code base which makes it even harder.

The comment should be added in the code base to make it more easy to understand 
and maintain.


> Add comment to HA code base
> ---
>
> Key: ROCKETMQ-173
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-173
> Project: Apache RocketMQ
>  Issue Type: Improvement
>  Components: rocketmq-store
>Reporter: Jaskey Lam
>Assignee: yukon
>
> The HA service is the very crucial part for rocketmq and the code base is 
> hard to understand.
> For now, it is lack of comment in the code base which makes it even harder.
> The comment should be added in the code base to make it  easier to understand 
> and maintain.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-106) Add flow control on topic level

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965907#comment-15965907
 ] 

ASF GitHub Bot commented on ROCKETMQ-106:
-

Github user lizhanhui commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/66
  
@vongosling @shroman any idea on this PR?


> Add flow control on topic level
> ---
>
> Key: ROCKETMQ-106
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-106
> Project: Apache RocketMQ
>  Issue Type: Wish
>  Components: rocketmq-client
>Reporter: Jaskey Lam
>Assignee: Jaskey Lam
>
> *Motivations*
> For current flow control, we can only control on queue level. 
> Howerver, the numbers of queue allocated may be dynamic changed. For example, 
> I might hope to control that at most 1000 messages can be pulled from broker 
> to protect my client. And I have no idea how many queue I am allocated. Maybe 
> I will have 5 queue and 5 instances so I set `pullThresholdForQueue`=1000, 
> which works as expected when one is fine. But as long as any instances 
> crashes, some instances may be allocated  more than one queue, which will 
> make messages pulled from broker exceed my expectations.
> A configuration of  `pullThresholdForTopic` is propably most user hopes.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (ROCKETMQ-163) DefaultMQPullConsumer.java批量拉取消息最大只能拉800个

2017-04-12 Thread Jaskey Lam (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965860#comment-15965860
 ] 

Jaskey Lam edited comment on ROCKETMQ-163 at 4/12/17 1:38 PM:
--

Please modify the tiltle in English and format the code in comment.

Also, clarify your problem please.


was (Author: jaskey):
Please modify the tiltle in English and format the code in comment in "{code}" 
block.

Also, clarify your problem please.

> DefaultMQPullConsumer.java批量拉取消息最大只能拉800个
> -
>
> Key: ROCKETMQ-163
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-163
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-store
>Affects Versions: 4.0.0-incubating
> Environment: linux
>Reporter: 郭中奇
>Assignee: yukon
> Fix For: 4.1.0-incubating
>
>
>  final int MaxFilterMessageCount = 16000;
> final boolean diskFallRecorded = 
> this.messageStoreConfig.isDiskFallRecorded();
> for (; i < bufferConsumeQueue.getSize() && i < 
> MaxFilterMessageCount; i += ConsumeQueue.CQStoreUnitSize) {
> long offsetPy = 
> bufferConsumeQueue.getByteBuffer().getLong();
> int sizePy = 
> bufferConsumeQueue.getByteBuffer().getInt();
> long tagsCode = 
> bufferConsumeQueue.getByteBuffer().getLong();
> maxPhyOffsetPulling = offsetPy;
> if (nextPhyFileStartOffset != Long.MIN_VALUE) {
> if (offsetPy < nextPhyFileStartOffset)
> continue;
> }



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-140) Register higher version broker against old name servers

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965890#comment-15965890
 ] 

ASF GitHub Bot commented on ROCKETMQ-140:
-

Github user lizhanhui closed the pull request at:

https://github.com/apache/incubator-rocketmq/pull/77


> Register higher version broker against old name servers
> ---
>
> Key: ROCKETMQ-140
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-140
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-namesrv
>Affects Versions: 4.1.0-incubating
>Reporter: Zhanhui Li
>Assignee: Zhanhui Li
>Priority: Minor
>
> When register higher version brokers again old name servers, 
> IndexOutOfBoundaryException may be thrown, causing registration failure.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-89) WS_DOMAIN_NAME, SUBGROUP default values overrides custom values passed by java options

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965913#comment-15965913
 ] 

ASF GitHub Bot commented on ROCKETMQ-89:


Github user lizhanhui commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/55
  
> But I don't quite understand why it has to have so many levels -- why not 
to remove rmqAddressServerDomain and let the user use only 
rocketmq.namesrv.domain ... ?

Yes, I am also considering centralizing default values to an external 
config-default.conf file and expose config.conf for user to override. This is 
best conducted in a dedicated PR after discussion.
For this PR, let's fix the logical fault first.


> WS_DOMAIN_NAME, SUBGROUP default values overrides custom values passed by 
> java options
> --
>
> Key: ROCKETMQ-89
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-89
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-broker
>Reporter: Zhanhui Li
>Assignee: Zhanhui Li
>Priority: Minor
> Fix For: 4.1.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-141) Make producers establish connection eagerly to new joined brokers

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965957#comment-15965957
 ] 

ASF GitHub Bot commented on ROCKETMQ-141:
-

Github user lizhanhui commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/78
  
Guys, any idea on this PR?


> Make producers establish connection eagerly to new joined brokers
> -
>
> Key: ROCKETMQ-141
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-141
> Project: Apache RocketMQ
>  Issue Type: Improvement
>  Components: rocketmq-client
>Affects Versions: 4.1.0-incubating
>Reporter: Zhanhui Li
>Assignee: Zhanhui Li
>
> When new brokers joins the cluster, MQ producer clients polls name server and 
> add the new-joined brokers to topicPublishInfo in case the new-joined broker 
> bears the topic. Later on, the producers may send messages to the new-joined 
> brokers.
> This achieves part of scalable goals and works fine for most cases. However, 
> we ran an issue in this process. The problem is when new broker joins the 
> cluster, the producer clients blocks for quite a while even if when 
> asynchronous send method is employed, which is very miserable for latency 
> sensitive scenarios.  
> After analyzing the root cause, it turns out that producer clients need to 
> establish a connection to the new-joined brokers the first time it sends a 
> message, as is blocking. 
> This issue is to establish a connection to new-joined brokers immediately 
> after polling name server and make them available to send methods thereafter. 
> Thus, when send is called against new-joined brokers, there is a writable 
> channel readily available.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (ROCKETMQ-174) Spelling Fix

2017-04-12 Thread Zhanhui Li (JIRA) 
Zhanhui Li created ROCKETMQ-174:
---

 Summary: Spelling Fix
 Key: ROCKETMQ-174
 URL: https://issues.apache.org/jira/browse/ROCKETMQ-174
 Project: Apache RocketMQ
  Issue Type: Improvement
Reporter: Zhanhui Li
Assignee: Zhanhui Li
Priority: Trivial


Existing code base contains many spelling issues, this is to fix them.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-174) Spelling Fix

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966188#comment-15966188
 ] 

ASF GitHub Bot commented on ROCKETMQ-174:
-

Github user coveralls commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/91
  

[![Coverage 
Status](https://coveralls.io/builds/11057354/badge)](https://coveralls.io/builds/11057354)

Coverage increased (+0.03%) to 34.653% when pulling 
**b5604b65a269e3ba191eb0aed83b485839b5011f on lizhanhui:ROCKETMQ-174** into 
**7bcb3b3eae1e3c441861f2a3cd79ff54a8e691b9 on apache:develop**.



> Spelling Fix
> 
>
> Key: ROCKETMQ-174
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-174
> Project: Apache RocketMQ
>  Issue Type: Improvement
>Reporter: Zhanhui Li
>Assignee: Zhanhui Li
>Priority: Trivial
>
> Existing code base contains many spelling issues, this is to fix them.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-174) Spelling Fix

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966187#comment-15966187
 ] 

ASF GitHub Bot commented on ROCKETMQ-174:
-

Github user coveralls commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/91
  

[![Coverage 
Status](https://coveralls.io/builds/11057354/badge)](https://coveralls.io/builds/11057354)

Coverage increased (+0.03%) to 34.653% when pulling 
**b5604b65a269e3ba191eb0aed83b485839b5011f on lizhanhui:ROCKETMQ-174** into 
**7bcb3b3eae1e3c441861f2a3cd79ff54a8e691b9 on apache:develop**.



> Spelling Fix
> 
>
> Key: ROCKETMQ-174
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-174
> Project: Apache RocketMQ
>  Issue Type: Improvement
>Reporter: Zhanhui Li
>Assignee: Zhanhui Li
>Priority: Trivial
>
> Existing code base contains many spelling issues, this is to fix them.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-174) Spelling Fix

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966186#comment-15966186
 ] 

ASF GitHub Bot commented on ROCKETMQ-174:
-

Github user coveralls commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/91
  

[![Coverage 
Status](https://coveralls.io/builds/11057354/badge)](https://coveralls.io/builds/11057354)

Coverage increased (+0.03%) to 34.653% when pulling 
**b5604b65a269e3ba191eb0aed83b485839b5011f on lizhanhui:ROCKETMQ-174** into 
**7bcb3b3eae1e3c441861f2a3cd79ff54a8e691b9 on apache:develop**.



> Spelling Fix
> 
>
> Key: ROCKETMQ-174
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-174
> Project: Apache RocketMQ
>  Issue Type: Improvement
>Reporter: Zhanhui Li
>Assignee: Zhanhui Li
>Priority: Trivial
>
> Existing code base contains many spelling issues, this is to fix them.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-174) Spelling Fix

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966164#comment-15966164
 ] 

ASF GitHub Bot commented on ROCKETMQ-174:
-

GitHub user lizhanhui opened a pull request:

https://github.com/apache/incubator-rocketmq/pull/91

[ROCKETMQ-174]Fix spelling errors

This is to fix large number of spelling errors.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lizhanhui/incubator-rocketmq ROCKETMQ-174

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rocketmq/pull/91.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #91


commit b5604b65a269e3ba191eb0aed83b485839b5011f
Author: Zhanhui Li 
Date:   2017-04-12T16:24:31Z

Spelling fix




> Spelling Fix
> 
>
> Key: ROCKETMQ-174
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-174
> Project: Apache RocketMQ
>  Issue Type: Improvement
>Reporter: Zhanhui Li
>Assignee: Zhanhui Li
>Priority: Trivial
>
> Existing code base contains many spelling issues, this is to fix them.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-27) Add storage partition online

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-27?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965721#comment-15965721
 ] 

ASF GitHub Bot commented on ROCKETMQ-27:


Github user lizhanhui commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/28
  
@Jaskey @vsair No, Broker only delete expired files.


> Add storage partition online
> 
>
> Key: ROCKETMQ-27
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-27
> Project: Apache RocketMQ
>  Issue Type: New Feature
>  Components: rocketmq-store, rocketmq-tools
>Affects Versions: 4.1.0-incubating
>Reporter: Zhanhui Li
>Assignee: vongosling
> Fix For: 4.1.0-incubating
>
>
> RocketMQ broker stores all messages before they are expired, which requires 
> pretty much storage for a high load system. For now, the message store can be 
> configured to use only one partition. Quite often, we find broker runs out of 
> storage as business expands quickly. Online resizing a partition most of time 
> is not possible(with absence of LVM) whilst attaching a new disk/partition in 
> cloud environment is super easy. 
> It would be nice for RocketMQ to be capable of adding extra commit log store 
> path online, allocating commit log files onto partitions based on 
> free/available space. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-140) Register higher version broker against old name servers

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965725#comment-15965725
 ] 

ASF GitHub Bot commented on ROCKETMQ-140:
-

Github user lizhanhui commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/77
  
As this PR is pretty trivial, I'll merge it soon.


> Register higher version broker against old name servers
> ---
>
> Key: ROCKETMQ-140
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-140
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-namesrv
>Affects Versions: 4.1.0-incubating
>Reporter: Zhanhui Li
>Assignee: Zhanhui Li
>Priority: Minor
>
> When register higher version brokers again old name servers, 
> IndexOutOfBoundaryException may be thrown, causing registration failure.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (ROCKETMQ-169) Support flow control when slave restarts and replicating from master

2017-04-12 Thread Jaskey Lam (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965784#comment-15965784
 ] 

Jaskey Lam edited comment on ROCKETMQ-169 at 4/12/17 1:09 PM:
--

I think this is an important and beneficial feature, since if a new slave or 
slave who is falling behind much starts, the out bandwidth will be occupied 
much and this will influence the consuming tps and synchronization of another 
healthy slave.

A trafic flow control should be done.

In my opinion, we can add the traffic control code logic in the master when 
process transfer data to slave :

{code}

processWriteEvent()...//existing trasfer logic
...
if (isSlaveRequestAVerySmallOffset) {
 computeTpsAndDoFlowControl();//sleep for while for this slave
} 

{code}

[~Yukon]
[~lizhanhui]
[~vongosling]

what's your advice?


was (Author: jaskey):
I think this is an important and beneficial feature, since if a new slave or 
slave who is falling behind much starts, the out bandwidth will be occupied 
much and this will influence the consuming tps and synchronization of another 
healthy slave.

A trafic flow control should be done.

In my opinion, we can add the traffic control code logic in the master when 
process reading data from slave for:

{code}

processWriteEvent()...//existing trasfer logic
...
if (isSlaveRequestAVerySmallOffset) {
 computeTpsAndDoFlowControl();//sleep for while for this slave
} 

{code}

[~Yukon]
[~lizhanhui]
[~vongosling]

what's your advice?

> Support flow control when slave restarts and replicating from master
> 
>
> Key: ROCKETMQ-169
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-169
> Project: Apache RocketMQ
>  Issue Type: Improvement
>  Components: rocketmq-broker
>Affects Versions: 4.0.0-incubating
>Reporter: Eason Chen
>Assignee: yukon
>
> as we know slave replicates message from master, but when slave restarts the 
> replicating will cause huge traffic which will impact sending messages, so we 
> suggest supporting flow control for slave when restarts.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-158) Remove logback dependency for rocketmq-tools

2017-04-12 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966245#comment-15966245
 ] 

ASF GitHub Bot commented on ROCKETMQ-158:
-

Github user Jaskey commented on a diff in the pull request:

https://github.com/apache/incubator-rocketmq/pull/85#discussion_r111210164
  
--- Diff: 
common/src/main/java/org/apache/rocketmq/common/utils/LogUtils.java ---
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.rocketmq.common.utils;
+
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.net.URL;
+import org.slf4j.ILoggerFactory;
+import org.slf4j.LoggerFactory;
+
+public class LogUtils {
+
+/**
+ * config logback dymatically from provided file path, otherwise try 
to config from resource file
--- End diff --

Thank you! I will update the commit soon.


> Remove logback dependency for rocketmq-tools
> 
>
> Key: ROCKETMQ-158
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-158
> Project: Apache RocketMQ
>  Issue Type: Improvement
>  Components: rocketmq-tools
>Reporter: Jaskey Lam
>Assignee: Jaskey Lam
>Priority: Minor
>
> Since user may need to use some admin interfaces to maintain something like 
> create topic, manage queues.
> They will need to use rocketmq-tools which contains DefaultMQAdminExt.
> But rocketmq-tools has explicitly depend on logback-classic and logback-core, 
> which may be conflict with the logging framework of the user's application.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)