[jira] [Commented] (ROCKETMQ-145) Hit ConcurrentModificationException in doWaitTransfer which happens very offen
[ https://issues.apache.org/jira/browse/ROCKETMQ-145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931031#comment-15931031 ] yukon commented on ROCKETMQ-145: And this issue also appears in GroupCommitService, which will be fixed together. > Hit ConcurrentModificationException in doWaitTransfer which happens very offen > -- > > Key: ROCKETMQ-145 > URL: https://issues.apache.org/jira/browse/ROCKETMQ-145 > Project: Apache RocketMQ > Issue Type: Bug > Components: rocketmq-store >Affects Versions: 4.0.0-incubating >Reporter: Eason Chen >Assignee: yukon > > we use master and slave , sync transfer and asyn flush, happens this very > offen: > 2017-03-17 20:12:38 WARN GroupTransferService - GroupTransferService service > has exception. > java.util.ConcurrentModificationException: null > at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901) > ~[na:1.8.0_121] > at java.util.ArrayList$Itr.next(ArrayList.java:851) ~[na:1.8.0_121] > at > org.apache.rocketmq.store.ha.HAService$GroupTransferService.doWaitTransfer(HAService.java:277) > ~[rocketmq-store-4.0.0-incubating.jar:4.0.0-incubating] > at > org.apache.rocketmq.store.ha.HAService$GroupTransferService.run(HAService.java:301) > ~[rocketmq-store-4.0.0-incubating.jar:4.0.0-incubating] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ROCKETMQ-145) Hit ConcurrentModificationException in doWaitTransfer which happens very offen
[ https://issues.apache.org/jira/browse/ROCKETMQ-145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931025#comment-15931025 ] Zhanhui Li commented on ROCKETMQ-145: - We have identified cause of this issue and will fix very soon. > Hit ConcurrentModificationException in doWaitTransfer which happens very offen > -- > > Key: ROCKETMQ-145 > URL: https://issues.apache.org/jira/browse/ROCKETMQ-145 > Project: Apache RocketMQ > Issue Type: Bug > Components: rocketmq-store >Affects Versions: 4.0.0-incubating >Reporter: Eason Chen >Assignee: yukon > > we use master and slave , sync transfer and asyn flush, happens this very > offen: > 2017-03-17 20:12:38 WARN GroupTransferService - GroupTransferService service > has exception. > java.util.ConcurrentModificationException: null > at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901) > ~[na:1.8.0_121] > at java.util.ArrayList$Itr.next(ArrayList.java:851) ~[na:1.8.0_121] > at > org.apache.rocketmq.store.ha.HAService$GroupTransferService.doWaitTransfer(HAService.java:277) > ~[rocketmq-store-4.0.0-incubating.jar:4.0.0-incubating] > at > org.apache.rocketmq.store.ha.HAService$GroupTransferService.run(HAService.java:301) > ~[rocketmq-store-4.0.0-incubating.jar:4.0.0-incubating] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ROCKETMQ-146) Hit "port out of range:" in decodeMessageId happens sometimes
Eason Chen created ROCKETMQ-146:
---
Summary: Hit "port out of range:" in decodeMessageId happens
sometimes
Key: ROCKETMQ-146
URL: https://issues.apache.org/jira/browse/ROCKETMQ-146
Project: Apache RocketMQ
Issue Type: Bug
Components: rocketmq-client, rocketmq-commons
Reporter: Eason Chen
Assignee: Xiaorui Wang
Attachments: {A99BE06A-745B-4FA8-859D-EFB6FB18E851}.png
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (ROCKETMQ-145) Hit ConcurrentModificationException in doWaitTransfer which happens very offen
Eason Chen created ROCKETMQ-145: --- Summary: Hit ConcurrentModificationException in doWaitTransfer which happens very offen Key: ROCKETMQ-145 URL: https://issues.apache.org/jira/browse/ROCKETMQ-145 Project: Apache RocketMQ Issue Type: Bug Components: rocketmq-store Affects Versions: 4.0.0-incubating Reporter: Eason Chen Assignee: yukon we use master and slave , sync transfer and asyn flush, happens this very offen: 2017-03-17 20:12:38 WARN GroupTransferService - GroupTransferService service has exception. java.util.ConcurrentModificationException: null at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901) ~[na:1.8.0_121] at java.util.ArrayList$Itr.next(ArrayList.java:851) ~[na:1.8.0_121] at org.apache.rocketmq.store.ha.HAService$GroupTransferService.doWaitTransfer(HAService.java:277) ~[rocketmq-store-4.0.0-incubating.jar:4.0.0-incubating] at org.apache.rocketmq.store.ha.HAService$GroupTransferService.run(HAService.java:301) ~[rocketmq-store-4.0.0-incubating.jar:4.0.0-incubating] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ROCKETMQ-80) Add batch feature
[ https://issues.apache.org/jira/browse/ROCKETMQ-80?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929758#comment-15929758 ] ASF GitHub Bot commented on ROCKETMQ-80: Github user vongosling commented on the issue: https://github.com/apache/incubator-rocketmq/pull/53 LGTM~ > Add batch feature > - > > Key: ROCKETMQ-80 > URL: https://issues.apache.org/jira/browse/ROCKETMQ-80 > Project: Apache RocketMQ > Issue Type: New Feature >Affects Versions: 4.1.0-incubating >Reporter: dongeforever >Assignee: dongeforever > Fix For: 4.1.0-incubating > > > Tests show that Kafka's million-level TPS is mainly owed to batch. When set > batch size to 1, the TPS is reduced an order of magnitude. So I try to add > this feature to RocketMQ. > For a minimal effort, it works as follows: > Only add synchronous send functions to MQProducer interface, just like > send(final Collection msgs). > Use MessageBatch which extends Message and implements Iterable. > Use byte buffer instead of list of objects to avoid too much GC in Broker. > Split the decode and encode logic from lockForPutMessage to avoid too many > race conditions. > Tests: > On linux with 24 Core 48G Ram and SSD, using 50 threads to send 50Byte(body) > message in batch size 50, we get about 150w TPS until the disk is full. > Potential problems: > Although the messages can be accumulated in the Broker very quickly, it need > time to dispatch to the consume queue, which is much slower than accepting > messages. So the messages may not be able to be consumed immediately. > We may need to refactor the ReputMessageService to solve this problem. > And if guys have some ideas, please let me know or just share it in this > issue. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
