[jira] [Commented] (ROCKETMQ-256) Manually clean disk to specified watermark with message consuming progress considered
[ https://issues.apache.org/jira/browse/ROCKETMQ-256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141451#comment-16141451 ] ASF GitHub Bot commented on ROCKETMQ-256: - Github user lizhanhui commented on the issue: https://github.com/apache/incubator-rocketmq/pull/143 @zhouxinyu @dongeforever please help review this PR. > Manually clean disk to specified watermark with message consuming progress > considered > - > > Key: ROCKETMQ-256 > URL: https://issues.apache.org/jira/browse/ROCKETMQ-256 > Project: Apache RocketMQ > Issue Type: Improvement > Components: rocketmq-broker, rocketmq-store, rocketmq-tools >Affects Versions: 4.1.0-incubating >Reporter: Zhanhui Li >Assignee: Zhanhui Li > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (ROCKETMQ-274) Some code may be can improvement
[
https://issues.apache.org/jira/browse/ROCKETMQ-274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangkai closed ROCKETMQ-274.
Resolution: Not A Problem
Different device has different cpu endians,we can use unsafe in single machine
, but the codec need to match different device ,so may be we need at least two
codec way for file system and network, but the improvement is not so clear for
much work.
> Some code may be can improvement
>
>
> Key: ROCKETMQ-274
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-274
> Project: Apache RocketMQ
> Issue Type: Improvement
> Components: rocketmq-store
>Affects Versions: 4.2.0-incubating
>Reporter: wangkai
>Assignee: yukon
>Priority: Minor
> Fix For: 4.2.0-incubating
>
>
> in class [CommitLog]
> I see many put Int or Long operation like this in function doAppend(...), We
> will result a better performance if we use DirectByteBuffer instead of
> HeapByteBuffer and use unsafe to operate the buffer.
> {code:java}
> // Initialization of storage space
> this.resetByteBuffer(msgStoreItemMemory, msgLen);
> // 1 TOTALSIZE
> this.msgStoreItemMemory.putInt(msgLen);
> // 2 MAGICCODE
> this.msgStoreItemMemory.putInt(CommitLog.MESSAGE_MAGIC_CODE);
> // 3 BODYCRC
> this.msgStoreItemMemory.putInt(msgInner.getBodyCRC());
> // 4 QUEUEID
> this.msgStoreItemMemory.putInt(msgInner.getQueueId());
> // 5 FLAG
> this.msgStoreItemMemory.putInt(msgInner.getFlag());
> // 6 QUEUEOFFSET
> this.msgStoreItemMemory.putLong(queueOffset);
> // 7 PHYSICALOFFSET
> this.msgStoreItemMemory.putLong(fileFromOffset +
> byteBuffer.position());
> // 8 SYSFLAG
> this.msgStoreItemMemory.putInt(msgInner.getSysFlag());
> // 9 BORNTIMESTAMP
> this.msgStoreItemMemory.putLong(msgInner.getBornTimestamp());
> // 10 BORNHOST
> this.resetByteBuffer(hostHolder, 8);
>
> this.msgStoreItemMemory.put(msgInner.getBornHostBytes(hostHolder));
> // 11 STORETIMESTAMP
> this.msgStoreItemMemory.putLong(msgInner.getStoreTimestamp());
> // 12 STOREHOSTADDRESS
> this.resetByteBuffer(hostHolder, 8);
>
> this.msgStoreItemMemory.put(msgInner.getStoreHostBytes(hostHolder));
> //this.msgBatchMemory.put(msgInner.getStoreHostBytes());
> // 13 RECONSUMETIMES
> this.msgStoreItemMemory.putInt(msgInner.getReconsumeTimes());
> // 14 Prepared Transaction Offset
>
> this.msgStoreItemMemory.putLong(msgInner.getPreparedTransactionOffset());
> // 15 BODY
> this.msgStoreItemMemory.putInt(bodyLength);
> if (bodyLength > 0)
> this.msgStoreItemMemory.put(msgInner.getBody());
> // 16 TOPIC
> this.msgStoreItemMemory.put((byte) topicLength);
> this.msgStoreItemMemory.put(topicData);
> // 17 PROPERTIES
> this.msgStoreItemMemory.putShort((short) propertiesLength);
> if (propertiesLength > 0)
> this.msgStoreItemMemory.put(propertiesData);
> {code}
> after the puts , at the line 1305:
> {code:java}
> byteBuffer.put(this.msgStoreItemMemory.array(), 0, msgLen);
> {code}
> it always put from zero of this buffer , but the resetByteBuffer function use
> flip() to "clean ?" the buffer , what if just do clear() the buffer in the
> function and limit it outside when need.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (ROCKETMQ-281) add check for preventing repeat start mq
yubaofu created ROCKETMQ-281: Summary: add check for preventing repeat start mq Key: ROCKETMQ-281 URL: https://issues.apache.org/jira/browse/ROCKETMQ-281 Project: Apache RocketMQ Issue Type: Improvement Components: rocketmq-broker Affects Versions: 4.1.0-incubating, 4.0.0-incubating Reporter: yubaofu Assignee: yukon Fix For: 4.2.0-incubating, 4.3.0-incubating add check for preventing repeat start mq -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ROCKETMQ-272) The config `syncFlushTimeout` doesn't work for SYNC_MASTER
[
https://issues.apache.org/jira/browse/ROCKETMQ-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141277#comment-16141277
]
ASF GitHub Bot commented on ROCKETMQ-272:
-
Github user evthoriz commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/153
@Jaskey Another key condition is the MASTER-SLAVE replicating mode, which
is SYNC_MASTER in the case.
However, the option `syncFlushTimeout` is useless not only for big
messages, but also those small messages. Small Messages just not trigger the
bug, but it still exists.
> The config `syncFlushTimeout` doesn't work for SYNC_MASTER
> --
>
> Key: ROCKETMQ-272
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-272
> Project: Apache RocketMQ
> Issue Type: Bug
> Components: rocketmq-broker
>Affects Versions: 4.1.0-incubating
>Reporter: Yu Kaiyuan
>Assignee: yukon
>
> It's quite frequent to get result as `sendStatus=FLUSH_SLAVE_TIMEOUT` when
> sending big messages(>500k) in SYNC_MASTER/SLAVE scenario.
> The timeout value used by the sync process currently as I found, is the
> config `syncFlushTimeout`. And its default value is 5000 milliseconds.
> But it shows that producer get the result as `FLUSH_SLAVE_TIMEOUT` less than
> 1 second.
> So why does the config not work as expected?
> Relevant code:
> {code:java}
> // CommitLog.java
> public void handleHA(AppendMessageResult result, PutMessageResult
> putMessageResult, MessageExt messageExt) {
> if (BrokerRole.SYNC_MASTER ==
> this.defaultMessageStore.getMessageStoreConfig().getBrokerRole()) {
> HAService service = this.defaultMessageStore.getHaService();
> if (messageExt.isWaitStoreMsgOK()) {
> // Determine whether to wait
> if (service.isSlaveOK(result.getWroteOffset() +
> result.getWroteBytes())) {
> GroupCommitRequest request = new
> GroupCommitRequest(result.getWroteOffset() + result.getWroteBytes());
> service.putRequest(request);
> service.getWaitNotifyObject().wakeupAll();
> boolean flushOK =
>
> request.waitForFlush(this.defaultMessageStore.getMessageStoreConfig().getSyncFlushTimeout());
> if (!flushOK) {
> log.error("do sync transfer other node, wait return, but
> failed, topic: " + messageExt.getTopic() + " tags: "
> + messageExt.getTags() + " client address: " +
> messageExt.getBornHostNameString());
>
> putMessageResult.setPutMessageStatus(PutMessageStatus.FLUSH_SLAVE_TIMEOUT);
> }
> }
> // Slave problem
> else {
> // Tell the producer, slave not available
>
> putMessageResult.setPutMessageStatus(PutMessageStatus.SLAVE_NOT_AVAILABLE);
> }
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ROCKETMQ-280) improve dependency management in parent pom
[
https://issues.apache.org/jira/browse/ROCKETMQ-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141245#comment-16141245
]
ASF GitHub Bot commented on ROCKETMQ-280:
-
Github user coveralls commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/155
[](https://coveralls.io/builds/12984817)
Coverage decreased (-0.07%) to 39.026% when pulling
**30f8898ef8a52b5183221ccd05dc9f7e35770285 on qqeasonchen:develop** into
**c6b2a998237615202ee9c4c6a1480c4a5d0eae59 on apache:develop**.
> improve dependency management in parent pom
>
>
> Key: ROCKETMQ-280
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-280
> Project: Apache RocketMQ
> Issue Type: Improvement
> Components: build
>Affects Versions: 4.1.0-incubating
> Environment: compile
>Reporter: Eason Chen
>Assignee: Stevens Chew
>Priority: Minor
> Fix For: 4.2.0-incubating
>
>
> {code:java}
> as is:
>
> org.apache.rocketmq
> rocketmq-filter
> ${project.version}
>
> to be:
>
> ${project.groupId}
> rocketmq-filter
> ${project.version}
>
>
> ${project.groupId}
> rocketmq-openmessaging
> ${project.version}
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ROCKETMQ-280) improve dependency management in parent pom
[
https://issues.apache.org/jira/browse/ROCKETMQ-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141246#comment-16141246
]
ASF GitHub Bot commented on ROCKETMQ-280:
-
Github user coveralls commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/155
[](https://coveralls.io/builds/12984817)
Coverage decreased (-0.07%) to 39.026% when pulling
**30f8898ef8a52b5183221ccd05dc9f7e35770285 on qqeasonchen:develop** into
**c6b2a998237615202ee9c4c6a1480c4a5d0eae59 on apache:develop**.
> improve dependency management in parent pom
>
>
> Key: ROCKETMQ-280
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-280
> Project: Apache RocketMQ
> Issue Type: Improvement
> Components: build
>Affects Versions: 4.1.0-incubating
> Environment: compile
>Reporter: Eason Chen
>Assignee: Stevens Chew
>Priority: Minor
> Fix For: 4.2.0-incubating
>
>
> {code:java}
> as is:
>
> org.apache.rocketmq
> rocketmq-filter
> ${project.version}
>
> to be:
>
> ${project.groupId}
> rocketmq-filter
> ${project.version}
>
>
> ${project.groupId}
> rocketmq-openmessaging
> ${project.version}
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
