[jira] [Commented] (ROCKETMQ-95) The config files of client log have been damaged

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-95?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882168#comment-15882168
 ] 

ASF GitHub Bot commented on ROCKETMQ-95:


GitHub user vsair opened a pull request:

https://github.com/apache/incubator-rocketmq/pull/70

[ROCKETMQ-95] fix damaged patterns

https://issues.apache.org/jira/browse/ROCKETMQ-95

Correct wrong patterns.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vsair/incubator-rocketmq ROCKETMQ-95

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-rocketmq/pull/70.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #70


commit 71bf2f3c55886f95fd770bc4a5e8064b05f50287
Author: xigu.lx 
Date:   2017-02-24T07:50:49Z

[ROCKETMQ-95] fix damaged patterns




> The config files of client log have been damaged
> 
>
> Key: ROCKETMQ-95
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-95
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-client
>Affects Versions: 4.0.0-incubating
>Reporter: Eric Liu
>Assignee: Xiaorui Wang
>  Labels: easyfix
> Fix For: 4.1.0-incubating
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> Config files:
> log4j_rocketmq_client.xml , logback_rocketmq_client.xml
> There is no log info printed at client, because config files are damaged, 
> such as:
> %properties, %defaultTopicQueueNums



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-102) When shutdown(), the persisted offet is not the latest consumed message, which may cause repeated messages

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882166#comment-15882166
 ] 

ASF GitHub Bot commented on ROCKETMQ-102:
-

Github user zhouxinyu commented on a diff in the pull request:

https://github.com/apache/incubator-rocketmq/pull/64#discussion_r102893550
  
--- Diff: 
client/src/main/java/org/apache/rocketmq/client/impl/consumer/ConsumeMessageOrderlyService.java
 ---
@@ -92,10 +92,22 @@ public void run() {
 }
 }
 
-public void shutdown() {
+@Override
+public void shutdown(long awaitTerminateMillis) {
 this.stopped = true;
 this.scheduledExecutorService.shutdown();
 this.consumeExecutor.shutdown();
+//await to consume
+if (awaitTerminateMillis > 0) {
+try {
+
this.consumeExecutor.awaitTermination(awaitTerminateMillis,TimeUnit.MILLISECONDS);
+if (!this.consumeExecutor.isTerminated()) log.info("There 
are messages still being consumed in thread pool, but not going to await them 
anymore. Have awaited for {} ms",awaitTerminateMillis);
--- End diff --

May be we need a common method to shutdown executor gracefully, like:

```
public static void shutdownGracefully(ExecutorService executor, long 
timeout, TimeUnit timeUnit) {
executor.shutdown();

try {
if(!executor.awaitTermination(timeout, timeUnit)) {
executor.shutdownNow();
if(!executor.awaitTermination(timeout, timeUnit)) {
LOG.warn(String.format("%s didn\'t terminate!", new 
Object[]{executor}));
}
}
} catch (InterruptedException var5) {
executor.shutdownNow();
Thread.currentThread().interrupt();
}

}
```


> When shutdown(), the persisted offet is not the latest consumed message, 
> which may cause repeated messages
> --
>
> Key: ROCKETMQ-102
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-102
> Project: Apache RocketMQ
>  Issue Type: Improvement
>  Components: rocketmq-client
>Reporter: Jaskey Lam
>Assignee: Xiaorui Wang
>
> When shutdown push consumer, push consumer will shutdwon thread pool then 
> persist offset.
> While shutdown thread pool is only stop submiting message to consume, which 
> does not stop consuming message which exists in the the thread queue or is 
> already being consumed.
> Which will cause repeated message very easily though user are shutdown 
> gracefully according to the provided interface.
> A way to solve this problem is needed. Such as accpet a param that how long 
> to wait for thread pool to terminated.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ROCKETMQ-110) consumeConcurrentlyMaxSpan has a very limited role

[Jaskey Lam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ROCKETMQ-110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jaskey Lam updated ROCKETMQ-110:

Description: 
Actually, in the current implementation of  consumeConcurrentlyMaxSpan , the 
role it plays is very limited.

In my opinion, RocketMQ hopes to solve a problem that if some messages blocks( 
may be possibly considered as dead lock or dead loop),say message with offset 
100 blocked,  and the rest whose offset is 101 to 2100 are very healthy, in the 
case that if the process is killed since it can not be shutdown normally if 
dead lock or dead loop is really happens when consuming, the later messages 101 
to 2100 which is consumed sucessfully will be repeated to consume again since 
the consumer offset will still remains at 100.

So to reduce the influence of repeated message numbers, flow control should be 
taken.  

But the current implementaion is to compare the span of last message of the 
first message to consumeConcurrentlyMaxSpan.

In the above example, the span is 2000 and flow control may do action to make 
it pause for one cycle for 50ms, but next time when the message 2100 and the 
rest of healthy message  consumed successfully , the fisrt key and the last key 
will be the same, 110, and the max span will be considered as 0, pull operation 
will continue.


public long getMaxSpan() {
try {
this.lockTreeMap.readLock().lockInterruptibly();
try {
if (!this.msgTreeMap.isEmpty()) {
return this.msgTreeMap.lastKey() - 
this.msgTreeMap.firstKey();
}
} finally {
this.lockTreeMap.readLock().unlock();
}
} catch (InterruptedException e) {
log.error("getMaxSpan exception", e);
}

return 0;
}

So for single message problem, consumeConcurrentlyMaxSpan will help nothing but 
in my opion this is what flow control should also takes into considerations.

I suggest maxSpan should be lastConsumedOffset(does not record now) - 
firstConsumingOffset(the first key of msgTreeMap).


  was:
Actually, in the current implementation of  consumeConcurrentlyMaxSpan , the 
role it plays is very limited.

In my opinion, RocketMQ hopes to solve a problem that if some messages blocks( 
may be possibly considered as dead lock or dead loop),say message with offset 
100 blocked,  and the rest whose offset is 101 to 2100 are very healthy, in the 
case that if the process is killed since it can not be shutdown normally if 
dead lock or dead loop is really happens when consuming, the later messages 101 
to 2100 which is consumed sucessfully will be repeated to consume again since 
the consumer offset will still remains at 100.

So to reduce the influence of repeated message numbers, flow control should be 
taken.  

But the current implementaion is to compare the span of last message of the 
first message to consumeConcurrentlyMaxSpan.

In the above example, the span is 2000 and flow control may do action to make 
it pause for one cycle for 50ms, but next time when the message 2100 and the 
rest of healthy message  consumed successfully , the fisrt key and the last key 
will be the same, 110, and the max span will be considered as 0, pull operation 
will continue.


public long getMaxSpan() {
try {
this.lockTreeMap.readLock().lockInterruptibly();
try {
if (!this.msgTreeMap.isEmpty()) {
return this.msgTreeMap.lastKey() - 
this.msgTreeMap.firstKey();
}
} finally {
this.lockTreeMap.readLock().unlock();
}
} catch (InterruptedException e) {
log.error("getMaxSpan exception", e);
}

return 0;
}

So for single message problem, consumeConcurrentlyMaxSpan will help nothing but 
in my opion this is what flow control should also takes into considerations.

I suggest maxSpan should be lastConsumedOffset - firstConsumingOffset(the first 
key of msgTreeMap).



> consumeConcurrentlyMaxSpan has a very limited role
> --
>
> Key: ROCKETMQ-110
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-110
> Project: Apache RocketMQ
>  Issue Type: Improvement
>  Components: rocketmq-client
>Affects Versions: 4.0.0-incubating
>Reporter: Jaskey Lam
>Assignee: Xiaorui Wang
>
> Actually, in the current implementation of  consumeConcurrentlyMaxSpan , the 
> role it plays is very limited.
> In my opinion, RocketMQ hopes to solve a problem that if some messages 
> blocks( may be possibly considered as dead lock or dead loop),say message 
> with offset 100 blocked,  and the rest whose offset is 101 to 2100 are very 
> healthy, in the case that if the process is killed since it can not be 
> 

[jira] [Created] (ROCKETMQ-110) consumeConcurrentlyMaxSpan has a very limited role

[Jaskey Lam (JIRA)]

Jaskey Lam created ROCKETMQ-110:
---

 Summary: consumeConcurrentlyMaxSpan has a very limited role
 Key: ROCKETMQ-110
 URL: https://issues.apache.org/jira/browse/ROCKETMQ-110
 Project: Apache RocketMQ
  Issue Type: Improvement
  Components: rocketmq-client
Affects Versions: 4.0.0-incubating
Reporter: Jaskey Lam
Assignee: Xiaorui Wang


Actually, in the current implementation of  consumeConcurrentlyMaxSpan , the 
role it plays is very limited.

In my opinion, RocketMQ hopes to solve a problem that if some messages blocks( 
may be possibly considered as dead lock or dead loop),say message with offset 
100 blocked,  and the rest whose offset is 101 to 2100 are very healthy, in the 
case that if the process is killed since it can not be shutdown normally if 
dead lock or dead loop is really happens when consuming, the later messages 101 
to 2100 which is consumed sucessfully will be repeated to consume again since 
the consumer offset will still remains at 100.

So to reduce the influence of repeated message numbers, flow control should be 
taken.  

But the current implementaion is to compare the span of last message of the 
first message to consumeConcurrentlyMaxSpan.

In the above example, the span is 2000 and flow control may do action to make 
it pause for one cycle for 50ms, but next time when the message 2100 and the 
rest of healthy message  consumed successfully , the fisrt key and the last key 
will be the same, 110, and the max span will be considered as 0, pull operation 
will continue.


public long getMaxSpan() {
try {
this.lockTreeMap.readLock().lockInterruptibly();
try {
if (!this.msgTreeMap.isEmpty()) {
return this.msgTreeMap.lastKey() - 
this.msgTreeMap.firstKey();
}
} finally {
this.lockTreeMap.readLock().unlock();
}
} catch (InterruptedException e) {
log.error("getMaxSpan exception", e);
}

return 0;
}

So for single message problem, consumeConcurrentlyMaxSpan will help nothing but 
in my opion this is what flow control should also takes into considerations.

I suggest maxSpan should be lastConsumedOffset - firstConsumingOffset(the first 
key of msgTreeMap).




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-106) Add flow control on topic level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882139#comment-15882139
 ] 

ASF GitHub Bot commented on ROCKETMQ-106:
-

Github user zhouxinyu commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/66
  
LGTM, but please add some unit tests at your convenience.
Please @vongosling @lizhanhui help review.


> Add flow control on topic level
> ---
>
> Key: ROCKETMQ-106
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-106
> Project: Apache RocketMQ
>  Issue Type: Wish
>  Components: rocketmq-client
>Reporter: Jaskey Lam
>Assignee: Jaskey Lam
>
> *Motivations*
> For current flow control, we can only control on queue level. 
> Howerver, the numbers of queue allocated may be dynamic changed. For example, 
> I might hope to control that at most 1000 messages can be pulled from broker 
> to protect my client. And I have no idea how many queue I am allocated. Maybe 
> I will have 5 queue and 5 instances so I set `pullThresholdForQueue`=1000, 
> which works as expected when one is fine. But as long as any instances 
> crashes, some instances may be allocated  more than one queue, which will 
> make messages pulled from broker exceed my expectations.
> A configuration of  `pullThresholdForTopic` is propably most user hopes.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-106) Add flow control on topic level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882129#comment-15882129
 ] 

ASF GitHub Bot commented on ROCKETMQ-106:
-

Github user zhouxinyu commented on a diff in the pull request:

https://github.com/apache/incubator-rocketmq/pull/66#discussion_r102890915
  
--- Diff: 
client/src/main/java/org/apache/rocketmq/client/consumer/DefaultMQPushConsumer.java
 ---
@@ -166,11 +166,16 @@
 private int consumeConcurrentlyMaxSpan = 2000;
 
 /**
- * Flow control threshold
+ * Flow control threshold on queue level
  */
 private int pullThresholdForQueue = 1000;
 
 /**
+ * Flow control threshold on topic level, should be greater or equals 
than pullThresholdForQueue
+ */
+private int pullThresholdForTopic = 2000;
--- End diff --

What will happen if we set a smaller `pullThresholdForTopic ` value than 
`pullThresholdForQueue`?


> Add flow control on topic level
> ---
>
> Key: ROCKETMQ-106
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-106
> Project: Apache RocketMQ
>  Issue Type: Wish
>  Components: rocketmq-client
>Reporter: Jaskey Lam
>Assignee: Jaskey Lam
>
> *Motivations*
> For current flow control, we can only control on queue level. 
> Howerver, the numbers of queue allocated may be dynamic changed. For example, 
> I might hope to control that at most 1000 messages can be pulled from broker 
> to protect my client. And I have no idea how many queue I am allocated. Maybe 
> I will have 5 queue and 5 instances so I set `pullThresholdForQueue`=1000, 
> which works as expected when one is fine. But as long as any instances 
> crashes, some instances may be allocated  more than one queue, which will 
> make messages pulled from broker exceed my expectations.
> A configuration of  `pullThresholdForTopic` is propably most user hopes.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-107) Access ServiceState is not thread safe when start() or shutdown()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881958#comment-15881958
 ] 

ASF GitHub Bot commented on ROCKETMQ-107:
-

Github user coveralls commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/68
  

[![Coverage 
Status](https://coveralls.io/builds/10309014/badge)](https://coveralls.io/builds/10309014)

Coverage increased (+0.1%) to 31.627% when pulling 
**57c2c6893b3fe1173aa7a68c15d63f309bd89138 on 
Jaskey:ROCKETMQ-107-synchroization-on-ServiceState** into 
**573b22c37806a21543b90707bcce6022243a62da on apache:master**.



> Access ServiceState is not thread safe when start() or shutdown()
> -
>
> Key: ROCKETMQ-107
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-107
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-client
>Reporter: Jaskey Lam
>Assignee: Jaskey Lam
>Priority: Minor
>
> When start() or shutdown(), service's state is not thread safe which may 
> break happen-before.
> For example: 
> switch (this.serviceState) {
> case CREATE_JUST:
> log.info("the consumer [{}] start beginning. messageModel={}, 
> isUnitMode={}", this.defaultMQPushConsumer.getConsumerGroup(),
> this.defaultMQPushConsumer.getMessageModel(), 
> this.defaultMQPushConsumer.isUnitMode());
> this.serviceState = ServiceState.START_FAILED;
> ..// do some start job here
> this.serviceState = ServiceState.RUNNING;
> break;
> case RUNNING:
> case START_FAILED:
> case SHUTDOWN_ALREADY:
> throw new MQClientException("The PushConsumer service state 
> not OK, maybe started once, "//
> + this.serviceState//
> + FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),
> null);
> default:
> break;
> }
> 1. If the user is start twice in two thread, the resources may initize twice.
> 2. if the user start in threadA and shutdown very quicky in another thread B, 
> shutdown may not reclaim the resources.
> Though the sceniro is very uncommon, but it is indeed a bug here. Fix is 
> actually quite trivial.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-107) Access ServiceState is not thread safe when start() or shutdown()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881957#comment-15881957
 ] 

ASF GitHub Bot commented on ROCKETMQ-107:
-

Github user coveralls commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/68
  

[![Coverage 
Status](https://coveralls.io/builds/10309014/badge)](https://coveralls.io/builds/10309014)

Coverage increased (+0.1%) to 31.627% when pulling 
**57c2c6893b3fe1173aa7a68c15d63f309bd89138 on 
Jaskey:ROCKETMQ-107-synchroization-on-ServiceState** into 
**573b22c37806a21543b90707bcce6022243a62da on apache:master**.



> Access ServiceState is not thread safe when start() or shutdown()
> -
>
> Key: ROCKETMQ-107
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-107
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-client
>Reporter: Jaskey Lam
>Assignee: Jaskey Lam
>Priority: Minor
>
> When start() or shutdown(), service's state is not thread safe which may 
> break happen-before.
> For example: 
> switch (this.serviceState) {
> case CREATE_JUST:
> log.info("the consumer [{}] start beginning. messageModel={}, 
> isUnitMode={}", this.defaultMQPushConsumer.getConsumerGroup(),
> this.defaultMQPushConsumer.getMessageModel(), 
> this.defaultMQPushConsumer.isUnitMode());
> this.serviceState = ServiceState.START_FAILED;
> ..// do some start job here
> this.serviceState = ServiceState.RUNNING;
> break;
> case RUNNING:
> case START_FAILED:
> case SHUTDOWN_ALREADY:
> throw new MQClientException("The PushConsumer service state 
> not OK, maybe started once, "//
> + this.serviceState//
> + FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),
> null);
> default:
> break;
> }
> 1. If the user is start twice in two thread, the resources may initize twice.
> 2. if the user start in threadA and shutdown very quicky in another thread B, 
> shutdown may not reclaim the resources.
> Though the sceniro is very uncommon, but it is indeed a bug here. Fix is 
> actually quite trivial.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-107) Access ServiceState is not thread safe when start() or shutdown()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881930#comment-15881930
 ] 

ASF GitHub Bot commented on ROCKETMQ-107:
-

Github user coveralls commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/68
  

[![Coverage 
Status](https://coveralls.io/builds/10308840/badge)](https://coveralls.io/builds/10308840)

Coverage increased (+0.3%) to 31.803% when pulling 
**57c2c6893b3fe1173aa7a68c15d63f309bd89138 on 
Jaskey:ROCKETMQ-107-synchroization-on-ServiceState** into 
**573b22c37806a21543b90707bcce6022243a62da on apache:master**.



> Access ServiceState is not thread safe when start() or shutdown()
> -
>
> Key: ROCKETMQ-107
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-107
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-client
>Reporter: Jaskey Lam
>Assignee: Jaskey Lam
>Priority: Minor
>
> When start() or shutdown(), service's state is not thread safe which may 
> break happen-before.
> For example: 
> switch (this.serviceState) {
> case CREATE_JUST:
> log.info("the consumer [{}] start beginning. messageModel={}, 
> isUnitMode={}", this.defaultMQPushConsumer.getConsumerGroup(),
> this.defaultMQPushConsumer.getMessageModel(), 
> this.defaultMQPushConsumer.isUnitMode());
> this.serviceState = ServiceState.START_FAILED;
> ..// do some start job here
> this.serviceState = ServiceState.RUNNING;
> break;
> case RUNNING:
> case START_FAILED:
> case SHUTDOWN_ALREADY:
> throw new MQClientException("The PushConsumer service state 
> not OK, maybe started once, "//
> + this.serviceState//
> + FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),
> null);
> default:
> break;
> }
> 1. If the user is start twice in two thread, the resources may initize twice.
> 2. if the user start in threadA and shutdown very quicky in another thread B, 
> shutdown may not reclaim the resources.
> Though the sceniro is very uncommon, but it is indeed a bug here. Fix is 
> actually quite trivial.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-107) Access ServiceState is not thread safe when start() or shutdown()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881951#comment-15881951
 ] 

ASF GitHub Bot commented on ROCKETMQ-107:
-

Github user coveralls commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/68
  

[![Coverage 
Status](https://coveralls.io/builds/10308981/badge)](https://coveralls.io/builds/10308981)

Coverage increased (+0.2%) to 31.72% when pulling 
**57c2c6893b3fe1173aa7a68c15d63f309bd89138 on 
Jaskey:ROCKETMQ-107-synchroization-on-ServiceState** into 
**573b22c37806a21543b90707bcce6022243a62da on apache:master**.



> Access ServiceState is not thread safe when start() or shutdown()
> -
>
> Key: ROCKETMQ-107
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-107
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-client
>Reporter: Jaskey Lam
>Assignee: Jaskey Lam
>Priority: Minor
>
> When start() or shutdown(), service's state is not thread safe which may 
> break happen-before.
> For example: 
> switch (this.serviceState) {
> case CREATE_JUST:
> log.info("the consumer [{}] start beginning. messageModel={}, 
> isUnitMode={}", this.defaultMQPushConsumer.getConsumerGroup(),
> this.defaultMQPushConsumer.getMessageModel(), 
> this.defaultMQPushConsumer.isUnitMode());
> this.serviceState = ServiceState.START_FAILED;
> ..// do some start job here
> this.serviceState = ServiceState.RUNNING;
> break;
> case RUNNING:
> case START_FAILED:
> case SHUTDOWN_ALREADY:
> throw new MQClientException("The PushConsumer service state 
> not OK, maybe started once, "//
> + this.serviceState//
> + FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),
> null);
> default:
> break;
> }
> 1. If the user is start twice in two thread, the resources may initize twice.
> 2. if the user start in threadA and shutdown very quicky in another thread B, 
> shutdown may not reclaim the resources.
> Though the sceniro is very uncommon, but it is indeed a bug here. Fix is 
> actually quite trivial.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-107) Access ServiceState is not thread safe when start() or shutdown()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881952#comment-15881952
 ] 

ASF GitHub Bot commented on ROCKETMQ-107:
-

Github user coveralls commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/68
  

[![Coverage 
Status](https://coveralls.io/builds/10308981/badge)](https://coveralls.io/builds/10308981)

Coverage increased (+0.2%) to 31.72% when pulling 
**57c2c6893b3fe1173aa7a68c15d63f309bd89138 on 
Jaskey:ROCKETMQ-107-synchroization-on-ServiceState** into 
**573b22c37806a21543b90707bcce6022243a62da on apache:master**.



> Access ServiceState is not thread safe when start() or shutdown()
> -
>
> Key: ROCKETMQ-107
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-107
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-client
>Reporter: Jaskey Lam
>Assignee: Jaskey Lam
>Priority: Minor
>
> When start() or shutdown(), service's state is not thread safe which may 
> break happen-before.
> For example: 
> switch (this.serviceState) {
> case CREATE_JUST:
> log.info("the consumer [{}] start beginning. messageModel={}, 
> isUnitMode={}", this.defaultMQPushConsumer.getConsumerGroup(),
> this.defaultMQPushConsumer.getMessageModel(), 
> this.defaultMQPushConsumer.isUnitMode());
> this.serviceState = ServiceState.START_FAILED;
> ..// do some start job here
> this.serviceState = ServiceState.RUNNING;
> break;
> case RUNNING:
> case START_FAILED:
> case SHUTDOWN_ALREADY:
> throw new MQClientException("The PushConsumer service state 
> not OK, maybe started once, "//
> + this.serviceState//
> + FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),
> null);
> default:
> break;
> }
> 1. If the user is start twice in two thread, the resources may initize twice.
> 2. if the user start in threadA and shutdown very quicky in another thread B, 
> shutdown may not reclaim the resources.
> Though the sceniro is very uncommon, but it is indeed a bug here. Fix is 
> actually quite trivial.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-107) Access ServiceState is not thread safe when start() or shutdown()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881953#comment-15881953
 ] 

ASF GitHub Bot commented on ROCKETMQ-107:
-

Github user coveralls commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/68
  

[![Coverage 
Status](https://coveralls.io/builds/10308981/badge)](https://coveralls.io/builds/10308981)

Coverage increased (+0.2%) to 31.72% when pulling 
**57c2c6893b3fe1173aa7a68c15d63f309bd89138 on 
Jaskey:ROCKETMQ-107-synchroization-on-ServiceState** into 
**573b22c37806a21543b90707bcce6022243a62da on apache:master**.



> Access ServiceState is not thread safe when start() or shutdown()
> -
>
> Key: ROCKETMQ-107
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-107
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-client
>Reporter: Jaskey Lam
>Assignee: Jaskey Lam
>Priority: Minor
>
> When start() or shutdown(), service's state is not thread safe which may 
> break happen-before.
> For example: 
> switch (this.serviceState) {
> case CREATE_JUST:
> log.info("the consumer [{}] start beginning. messageModel={}, 
> isUnitMode={}", this.defaultMQPushConsumer.getConsumerGroup(),
> this.defaultMQPushConsumer.getMessageModel(), 
> this.defaultMQPushConsumer.isUnitMode());
> this.serviceState = ServiceState.START_FAILED;
> ..// do some start job here
> this.serviceState = ServiceState.RUNNING;
> break;
> case RUNNING:
> case START_FAILED:
> case SHUTDOWN_ALREADY:
> throw new MQClientException("The PushConsumer service state 
> not OK, maybe started once, "//
> + this.serviceState//
> + FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),
> null);
> default:
> break;
> }
> 1. If the user is start twice in two thread, the resources may initize twice.
> 2. if the user start in threadA and shutdown very quicky in another thread B, 
> shutdown may not reclaim the resources.
> Though the sceniro is very uncommon, but it is indeed a bug here. Fix is 
> actually quite trivial.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-107) Access ServiceState is not thread safe when start() or shutdown()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881975#comment-15881975
 ] 

ASF GitHub Bot commented on ROCKETMQ-107:
-

Github user coveralls commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/68
  

[![Coverage 
Status](https://coveralls.io/builds/10309078/badge)](https://coveralls.io/builds/10309078)

Coverage increased (+0.1%) to 31.636% when pulling 
**57c2c6893b3fe1173aa7a68c15d63f309bd89138 on 
Jaskey:ROCKETMQ-107-synchroization-on-ServiceState** into 
**573b22c37806a21543b90707bcce6022243a62da on apache:master**.



> Access ServiceState is not thread safe when start() or shutdown()
> -
>
> Key: ROCKETMQ-107
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-107
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-client
>Reporter: Jaskey Lam
>Assignee: Jaskey Lam
>Priority: Minor
>
> When start() or shutdown(), service's state is not thread safe which may 
> break happen-before.
> For example: 
> switch (this.serviceState) {
> case CREATE_JUST:
> log.info("the consumer [{}] start beginning. messageModel={}, 
> isUnitMode={}", this.defaultMQPushConsumer.getConsumerGroup(),
> this.defaultMQPushConsumer.getMessageModel(), 
> this.defaultMQPushConsumer.isUnitMode());
> this.serviceState = ServiceState.START_FAILED;
> ..// do some start job here
> this.serviceState = ServiceState.RUNNING;
> break;
> case RUNNING:
> case START_FAILED:
> case SHUTDOWN_ALREADY:
> throw new MQClientException("The PushConsumer service state 
> not OK, maybe started once, "//
> + this.serviceState//
> + FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),
> null);
> default:
> break;
> }
> 1. If the user is start twice in two thread, the resources may initize twice.
> 2. if the user start in threadA and shutdown very quicky in another thread B, 
> shutdown may not reclaim the resources.
> Though the sceniro is very uncommon, but it is indeed a bug here. Fix is 
> actually quite trivial.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-107) Access ServiceState is not thread safe when start() or shutdown()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881973#comment-15881973
 ] 

ASF GitHub Bot commented on ROCKETMQ-107:
-

Github user coveralls commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/68
  

[![Coverage 
Status](https://coveralls.io/builds/10309078/badge)](https://coveralls.io/builds/10309078)

Coverage increased (+0.1%) to 31.636% when pulling 
**57c2c6893b3fe1173aa7a68c15d63f309bd89138 on 
Jaskey:ROCKETMQ-107-synchroization-on-ServiceState** into 
**573b22c37806a21543b90707bcce6022243a62da on apache:master**.



> Access ServiceState is not thread safe when start() or shutdown()
> -
>
> Key: ROCKETMQ-107
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-107
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-client
>Reporter: Jaskey Lam
>Assignee: Jaskey Lam
>Priority: Minor
>
> When start() or shutdown(), service's state is not thread safe which may 
> break happen-before.
> For example: 
> switch (this.serviceState) {
> case CREATE_JUST:
> log.info("the consumer [{}] start beginning. messageModel={}, 
> isUnitMode={}", this.defaultMQPushConsumer.getConsumerGroup(),
> this.defaultMQPushConsumer.getMessageModel(), 
> this.defaultMQPushConsumer.isUnitMode());
> this.serviceState = ServiceState.START_FAILED;
> ..// do some start job here
> this.serviceState = ServiceState.RUNNING;
> break;
> case RUNNING:
> case START_FAILED:
> case SHUTDOWN_ALREADY:
> throw new MQClientException("The PushConsumer service state 
> not OK, maybe started once, "//
> + this.serviceState//
> + FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),
> null);
> default:
> break;
> }
> 1. If the user is start twice in two thread, the resources may initize twice.
> 2. if the user start in threadA and shutdown very quicky in another thread B, 
> shutdown may not reclaim the resources.
> Though the sceniro is very uncommon, but it is indeed a bug here. Fix is 
> actually quite trivial.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-107) Access ServiceState is not thread safe when start() or shutdown()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881974#comment-15881974
 ] 

ASF GitHub Bot commented on ROCKETMQ-107:
-

Github user coveralls commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/68
  

[![Coverage 
Status](https://coveralls.io/builds/10309078/badge)](https://coveralls.io/builds/10309078)

Coverage increased (+0.1%) to 31.636% when pulling 
**57c2c6893b3fe1173aa7a68c15d63f309bd89138 on 
Jaskey:ROCKETMQ-107-synchroization-on-ServiceState** into 
**573b22c37806a21543b90707bcce6022243a62da on apache:master**.



> Access ServiceState is not thread safe when start() or shutdown()
> -
>
> Key: ROCKETMQ-107
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-107
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-client
>Reporter: Jaskey Lam
>Assignee: Jaskey Lam
>Priority: Minor
>
> When start() or shutdown(), service's state is not thread safe which may 
> break happen-before.
> For example: 
> switch (this.serviceState) {
> case CREATE_JUST:
> log.info("the consumer [{}] start beginning. messageModel={}, 
> isUnitMode={}", this.defaultMQPushConsumer.getConsumerGroup(),
> this.defaultMQPushConsumer.getMessageModel(), 
> this.defaultMQPushConsumer.isUnitMode());
> this.serviceState = ServiceState.START_FAILED;
> ..// do some start job here
> this.serviceState = ServiceState.RUNNING;
> break;
> case RUNNING:
> case START_FAILED:
> case SHUTDOWN_ALREADY:
> throw new MQClientException("The PushConsumer service state 
> not OK, maybe started once, "//
> + this.serviceState//
> + FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),
> null);
> default:
> break;
> }
> 1. If the user is start twice in two thread, the resources may initize twice.
> 2. if the user start in threadA and shutdown very quicky in another thread B, 
> shutdown may not reclaim the resources.
> Though the sceniro is very uncommon, but it is indeed a bug here. Fix is 
> actually quite trivial.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-102) When shutdown(), the persisted offet is not the latest consumed message, which may cause repeated messages

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882160#comment-15882160
 ] 

ASF GitHub Bot commented on ROCKETMQ-102:
-

Github user zhouxinyu commented on a diff in the pull request:

https://github.com/apache/incubator-rocketmq/pull/64#discussion_r102893082
  
--- Diff: 
client/src/test/java/org/apache/rocketmq/client/consumer/DefaultMQPushConsumerTest.java
 ---
@@ -52,6 +56,7 @@
 import org.apache.rocketmq.common.protocol.header.PullMessageRequestHeader;
 import org.apache.rocketmq.remoting.exception.RemotingException;
 import org.junit.After;
+import org.junit.Assert;
--- End diff --

Hi, let's unify the assert tool and use 
`org.assertj.core.api.Assertions.assertThat`.


> When shutdown(), the persisted offet is not the latest consumed message, 
> which may cause repeated messages
> --
>
> Key: ROCKETMQ-102
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-102
> Project: Apache RocketMQ
>  Issue Type: Improvement
>  Components: rocketmq-client
>Reporter: Jaskey Lam
>Assignee: Xiaorui Wang
>
> When shutdown push consumer, push consumer will shutdwon thread pool then 
> persist offset.
> While shutdown thread pool is only stop submiting message to consume, which 
> does not stop consuming message which exists in the the thread queue or is 
> already being consumed.
> Which will cause repeated message very easily though user are shutdown 
> gracefully according to the provided interface.
> A way to solve this problem is needed. Such as accpet a param that how long 
> to wait for thread pool to terminated.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ROCKETMQ-107) Access ServiceState is not thread safe when start() or shutdown()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ROCKETMQ-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880306#comment-15880306
 ] 

ASF GitHub Bot commented on ROCKETMQ-107:
-

Github user Ah39 commented on the issue:

https://github.com/apache/incubator-rocketmq/pull/68
  
1.AtomicReference State = new AtomicReference()
state.compareAndSet

2.volatile

can fix the problem 


> Access ServiceState is not thread safe when start() or shutdown()
> -
>
> Key: ROCKETMQ-107
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-107
> Project: Apache RocketMQ
>  Issue Type: Bug
>  Components: rocketmq-client
>Reporter: Jaskey Lam
>Assignee: Jaskey Lam
>Priority: Minor
>
> When start() or shutdown(), service's state is not thread safe which may 
> break happen-before.
> For example: 
> switch (this.serviceState) {
> case CREATE_JUST:
> log.info("the consumer [{}] start beginning. messageModel={}, 
> isUnitMode={}", this.defaultMQPushConsumer.getConsumerGroup(),
> this.defaultMQPushConsumer.getMessageModel(), 
> this.defaultMQPushConsumer.isUnitMode());
> this.serviceState = ServiceState.START_FAILED;
> ..// do some start job here
> this.serviceState = ServiceState.RUNNING;
> break;
> case RUNNING:
> case START_FAILED:
> case SHUTDOWN_ALREADY:
> throw new MQClientException("The PushConsumer service state 
> not OK, maybe started once, "//
> + this.serviceState//
> + FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),
> null);
> default:
> break;
> }
> 1. If the user is start twice in two thread, the resources may initize twice.
> 2. if the user start in threadA and shutdown very quicky in another thread B, 
> shutdown may not reclaim the resources.
> Though the sceniro is very uncommon, but it is indeed a bug here. Fix is 
> actually quite trivial.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)