cserwen opened a new issue #3787:
URL: https://github.com/apache/rocketmq/issues/3787
**BUG REPORT**
### Please describe the issue you observed:
- What did you do (The steps to reproduce)?
> Use pop model to consume and the params is as fllows:
```bash
#client
consumeThreadMin: 1
consumeThreadMax: 1
popBatchNums: 32 (default value)
popThresholdForQueue: 96 (default value)
# broker
enablePopBufferMerge: true
enablePopLog: false
Others is all default value
```
> And the message listener is:
``` java
msgs.forEach(System.out::println);
try {
Thread.sleep(100);
} catch (Exception e) {
e.printStackTrace();
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
```
> **So the consume qps is 10 for each client.**
- What did you expect to see?
> It should only be consumed once normally.
- What did you see instead?
> There are many messages that are considered to have failed to be consumed
but are actually consumed successfully. So these messages are written to the
retry queue and thus consumed again
### Please tell us about your environment:
> 5.0.0-alpha
### Other information (e.g. detailed explanation, logs, related issues,
suggestions how to fix, etc):
This problem occurs when ck messages are written from buffer to store. When
the ck message stays in the buffer for more than 8.5 seconds, the corresponding
ack message can only be written to the store.
https://github.com/apache/rocketmq/blob/52548056cc85381bf774e913c168ef8f1bd810ee/broker/src/main/java/org/apache/rocketmq/broker/processor/PopBufferMergeService.java#L494
So the ack will be written to the topic firstly. The ck msg will be written
when the time in the buffer exceeds 10s.
https://github.com/apache/rocketmq/blob/52548056cc85381bf774e913c168ef8f1bd810ee/broker/src/main/java/org/apache/rocketmq/broker/processor/PopBufferMergeService.java#L236
Because the ack and ck messages are in the same delay level, the previous
ones will reach the `reviveTopic` firstly. This will cause the ack message to
be delivered to `reviveTopic` earlier than the ck message.So `reviveService`
thinks that the message has not been acked.
**If we can support second-level delay messages, this problem can be easily
solved.** Because the ck msg can be delivered earlier.
https://github.com/apache/rocketmq/blob/52548056cc85381bf774e913c168ef8f1bd810ee/broker/src/main/java/org/apache/rocketmq/broker/processor/PopMessageProcessor.java#L662
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
