wenxuwan opened a new pull request, #1245: URL: https://github.com/apache/rocketmq-client-go/pull/1245
## What is the purpose of the change Prevent push consumers from pulling with stale/default queue thresholds while rebalance is still initializing the remaining queues. Currently, each queue's pull request is dispatched immediately after its offset lookup, but topic-derived thresholds are applied only after the initialization loop returns. With a backlog and slow offset lookups, earlier queues can pull multiple batches using the default 1024-message / 512-MiB limits. For example, a 5000-message / 64-MiB topic budget across 64 queues should produce 78-message / 1-MiB queue thresholds before those queues begin pulling. ## Brief changelog - Collect new pull requests while initializing process queues, update queue flow control, and then dispatch the requests. - Keep subscription version updates and broker heartbeats in `messageQueueChanged`. - Dispose of undispatched process queues when shutdown interrupts initialization or dispatch, without removing a replacement queue. - Add regression coverage for dispatch ordering, queue additions/removals, offset lookup failures, shutdown, threshold semantics, orderly locking, both message models, and pull-consumer compatibility. This preserves the existing threshold calculation and batch-level flow-control behavior. It does not introduce a hard global memory limit or include temporary diagnostic logging. ## Verifying this change - `go test -race ./consumer -run '^TestRebalance' -count=5 -timeout=120s` passed. - All 46 consumer tests other than the existing five-minute `TestNewStatsManager` passed with Go 1.18.10. - The new dispatch-order regression test previously failed against the pre-fix implementation and passed with the fix. - Application-level startup comparison with 64 business queues, 32-KiB bodies, and topic thresholds of 5000 messages / 64 MiB: before the fix, 928 successful batches used the default thresholds during a roughly 3.65-second startup window; after the fix, none of the 641 recorded successful batches used the defaults. The two runs used different consumption offsets, so this validates the ordering change rather than a precise RSS reduction. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
