VGalaxies commented on code in PR #19340:
URL: https://github.com/apache/iotdb/pull/19340#discussion_r4640037450
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusPrefetchingQueue.java:
##########
@@ -4873,6 +4879,8 @@ private void applySeekResetUnderWriteLock(final
PendingSeekRequest request) {
writerStates.clear();
clearRecoveryWriterProgress();
materializedProgressByWriter.clear();
+ commitManager.claimTransientInFlightStateIfPresent(
Review Comment:
Thanks for catching this! Addressed in 204c42c626: removed the unconditional
`claimTransientInFlightStateIfPresent` call and added an owner-fenced
`ConsensusSubscriptionCommitManager.resetState(..., owner)` overload that
atomically checks `state.isSupersededTransientOwner(owner)` inside
`synchronized (state)`. When superseded, `resetState` returns `null` without
taking ownership, clearing replacement mappings, updating the committed
frontier, or persisting, and `ConsensusPrefetchingQueue` aborts the seek
cleanly by failing the pending seek request. Added regression tests in
`ConsensusSubscriptionCommitStateTest` and
`ConsensusPrefetchingQueueSeekTest#testSupersededQueueSeekIsRejectedWithoutOverwritingReplacementState`.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusPrefetchingQueue.java:
##########
@@ -1093,6 +1093,9 @@ private boolean initPrefetchUnderInitializationLock(final
RegionProgress regionP
commitManager.isConfigNodeProgressAvailable(consumerGroupId,
topicName, consensusGroupId);
final boolean hasPersistedState =
commitManager.hasPersistedState(consumerGroupId, topicName,
consensusGroupId);
+ // Claim transient commit ownership and discard any stale in-flight state
left by a previous
+ // queue or deactivation cycle before snapshotting committed progress and
starting replay.
+ commitManager.claimTransientInFlightState(consumerGroupId, topicName,
consensusGroupId, this);
Review Comment:
Kept as-is: `resolveCommittedRegionProgressForInit()` on the very next line
already calls `commitManager.getOrCreateState(...)` unconditionally, so
`claimTransientInFlightState` does not cause extra state materialization, and
claiming ownership during initialization is required to fence any superseded
queue before replay begins.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusSubscriptionCommitManager.java:
##########
@@ -2076,6 +2149,12 @@ protected boolean removeEldestEntry(final
Map.Entry<ProgressKey, Boolean> eldest
private final Map<Integer, NavigableSet<ProgressKey>>
committedPendingKeysByWriter =
new HashMap<>();
+ /**
+ * Active queue instance owning transient in-flight mappings so delayed
async cleanup or late
+ * prefetch/commit calls from a superseded queue cannot corrupt a
replacement queue.
+ */
+ private Object transientOwner;
Review Comment:
Addressed in 204c42c626: `ConsensusPrefetchingQueue` now passes a
lightweight opaque token (`private final Object transientCommitOwnerToken = new
Object()`) instead of `this`, so `ConsensusSubscriptionCommitState` never
retains a strong reference to closed queue instances.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusSubscriptionCommitManager.java:
##########
@@ -2084,6 +2163,59 @@ public ConsensusSubscriptionCommitState(
syncPersistedProgress();
}
+ public void clearTransientInFlightState() {
+ synchronized (this) {
+ clearTransientInFlightStateUnderLock();
+ transientOwner = null;
+ }
+ }
+
+ void claimTransientInFlightState(final Object owner) {
+ synchronized (this) {
+ clearTransientInFlightStateUnderLock();
+ transientOwner = owner;
+ }
+ }
+
+ boolean clearTransientInFlightStateIfOwner(final Object owner) {
+ synchronized (this) {
+ if (Objects.nonNull(owner) && Objects.nonNull(transientOwner) &&
transientOwner != owner) {
+ return false;
+ }
+ clearTransientInFlightStateUnderLock();
+ transientOwner = null;
+ return true;
+ }
+ }
Review Comment:
Addressed in 204c42c626: tightened `clearTransientInFlightStateIfOwner` to
reject `owner == null` whenever `transientOwner != null`
(`Objects.nonNull(transientOwner) && transientOwner != owner`), keeping
`clearTransientInFlightState()` as the explicit unfenced clear API, and added
test coverage in `ConsensusSubscriptionCommitStateTest`.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusPrefetchingQueue.java:
##########
@@ -3434,8 +3437,10 @@ private boolean createAndEnqueueEvent(
final SubscriptionCommitContext commitContext =
buildWriterCommitContext(commitLocalSeq);
final WriterId writerId = commitContext.getWriterId();
final WriterProgress writerProgress = commitContext.getWriterProgress();
- commitManager.recordMapping(
- consumerGroupId, topicName, consensusGroupId, writerId,
writerProgress);
+ if (!commitManager.recordMapping(
+ consumerGroupId, topicName, consensusGroupId, writerId,
writerProgress, this)) {
Review Comment:
Skipped: `createAndEnqueueEvent` is on the hot prefetch path and this does
not affect correctness or recovery semantics, so avoiding extra logging (and
new EN/ZH i18n message constants) keeps the fix minimal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]