qianye1001 opened a new pull request, #1326:
URL: https://github.com/apache/rocketmq-clients/pull/1326
### Which Issue(s) This PR Fixes
Fixes #1325
### Brief Description
The Java gRPC PushConsumer currently sends one acknowledgement RPC for every
successfully consumed message even though `AckMessageRequest` supports repeated
entries. Under a high consume rate, these one-entry requests add substantial
client and Proxy overhead.
This change introduces one acknowledgement batcher per PushConsumer:
- Batches are partitioned by endpoint and topic to match the request routing
and protocol constraints.
- A batch is flushed after 5 seconds or when it reaches 1,024 messages.
- Only messages consumed within 25 seconds of receive/decode are eligible.
Slow messages use the existing immediate acknowledgement path.
- FIFO acknowledgements remain immediate because subsequent ordered messages
depend on them.
- Lite consumers retain the existing immediate acknowledgement behavior.
- Multi-entry responses are mapped back to each message by message ID.
Missing or failed results use the existing delayed individual retry path;
invalid receipt handles preserve the existing no-retry behavior.
- PushConsumer shutdown waits for consumption to finish and then flushes
pending and in-flight acknowledgement batches.
- ACK interceptors still receive all messages included in each batch.
The current RocketMQ Proxy protocol and batch-ACK path already accept
repeated entries, so this change does not require a protocol update.
At approximately 8,000 TPS in the medium 2B workload, two controlled
candidate runs averaged about 20.4 entries per ACK RPC, reducing ACK RPC volume
by approximately 95.1%. Total Java consumer CPU decreased by approximately
20.6%, while throughput remained unchanged. A separate single-topic validation
observed requests with exactly 1,024 entries.
### How Did You Test This Change?
- `zsh -ic 'javaswitch 11 >/dev/null && mvn -f java/pom.xml -pl client -am
test'`
- 304 tests run
- 0 failures
- 0 errors
- 1 skipped
- Checkstyle: 0 violations
- SpotBugs: 0 errors or warnings
- Verified the compiled batcher class remains Java 8 compatible (`major
version: 52`).
- Ran an end-to-end Kubernetes smoke test against a two-Broker/two-Proxy
deployment with batch ACK enabled.
- Ran a medium 2B A-B-B-A comparison with 1,000 topics, 1,000 groups, 2,000
producers, 2,000 consumers, 4 KiB messages, 120 seconds warm-up, and 300
seconds sampling per round.
| Metric | Baseline mean | Batch ACK mean | Change |
| --- | ---: | ---: | ---: |
| Send TPS | 7,981.0 | 7,981.5 | +0.01% |
| Consumer total CPU | 3.446 cores | 2.736 cores | -20.59% |
| Consumer CPU p95 | 3.658 cores | 2.975 cores | -18.67% |
| Proxy total CPU | 10.998 cores | 9.052 cores | -17.70% |
| E2E latency p99 | 21.682 ms | 20.515 ms | -5.38% |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
