ai-yang opened a new pull request, #10701:
URL: https://github.com/apache/rocketmq/pull/10701
### Which Issue(s) This PR Fixes
- Fixes #10700
### Brief Description
Make first offset-map initialization atomic in the classic and RocksDB v1
consumer offset managers, and serialize incremental whole-map persistence for
the same `topic@group`.
- Publish one shared inner queue-offset map with outer `putIfAbsent`, then
update the selected winner map.
- Preserve the RocksDB v1 LMQ-specific map construction.
- In incremental mode, order the queue update, version update, whole-map
serialization, and WAL batch write under the inner map monitor.
- Keep non-incremental commits lock-free across the existing concurrent map
and allow different `topic@group` keys to proceed independently.
- Leave RocksDB v2 unchanged because it already initializes atomically and
persists each queue under an independent key.
### Root Cause
Classic and RocksDB v1 used check-then-act initialization followed by
unconditional outer `put`, so two first commits could publish different inner
maps and overwrite one queue. RocksDB v1 incremental mode additionally allowed
an older serialized whole-map batch to be written after a newer batch for the
same RocksDB key.
Historical unmerged PR #1427 identified the classic initialization race.
This PR preserves that analysis while extending coverage to RocksDB v1,
incremental WAL ordering, and deterministic persistence/reload tests. Open PRs
#10625, #9602, and #9877 touch related files but do not change these commit
paths.
### Impact
Concurrent first commits retain every queue in memory and after persistence.
Incremental restart recovery cannot regress to an older same-key snapshot.
Existing rollback warnings, version semantics, LMQ allocation, and v2 behavior
are preserved.
### How Did You Test This Change?
- Deterministic red tests on the previous implementation:
- classic concurrent first commits: expected 2 queues, reloaded 1;
- RocksDB v1 periodic LMQ commits: expected 2 queues, retained 1;
- RocksDB v1 incremental delayed old batch: expected 2 queues after
restart, reloaded 1.
- `ConsumerOffsetManagerTest` + `RocksDBConsumerOffsetManagerTest`: 21 tests
passed.
- Full `broker` test suite: 755 tests passed, 0 failures, 0 errors, 4
skipped.
- Maven Checkstyle: 0 violations.
- SpotBugs: 0 findings.
- `git diff --check`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
