VGalaxies opened a new pull request, #19340: URL: https://github.com/apache/iotdb/pull/19340
## Problem When a `ConsensusPrefetchingQueue` closes, deactivates, or is replaced by a new queue instance (for example after consumer disconnection, failover, or restart), transient in-flight tracking state inside `ConsensusSubscriptionCommitState` (`outstandingKeys`, `outstandingKeysByWriter`, `committedPendingKeysByWriter`, and uncommitted entries in `recentlyCommittedKeys`) remained intact across queue lifecycles. When a replacement queue replayed from the committed frontier and re-batched WAL entries into different batch boundaries: 1. Old uncommitted `outstandingKeys` from the previous queue lifecycle could never be acknowledged again, pinning the contiguous commit watermark (`committedWriterPositions`) and freezing commit progress. 2. Uncommitted out-of-order ACKs retained in `recentlyCommittedKeys` could interfere with newly mapped replayed batches. 3. Delayed async cleanup, mapping, or commit calls from a superseded queue could corrupt the live transient state of the replacement queue. ## Fix - Add transient owner fencing (`transientOwner`) and transient in-flight state lifecycle methods (`claimTransientInFlightState`, `clearTransientInFlightStateIfOwner`, `clearTransientInFlightState`) on `ConsensusSubscriptionCommitState` / `ConsensusSubscriptionCommitManager`. - Prune uncommitted entries (`> committedKey`) from `recentlyCommittedKeys` when clearing transient in-flight state while retaining already-committed keys for idempotent re-commit. - Enforce `transientOwner` fencing atomically inside `ConsensusSubscriptionCommitState`'s monitor (`synchronized (this)`) for `recordMapping`, `commit`, and `commitWithoutOutstanding`, preventing TOCTOU races between owner validation and state mutation. - Claim transient ownership in `ConsensusPrefetchingQueue` during initialization and seek reset, and clear transient state for the current owner on close and deactivation. ## Tests - `ConsensusSubscriptionCommitStateTest`: added `testClearTransientInFlightStateUnblocksRebatchedReplayAfterRestart` and `testTransientOwnerFencingProtectsReplacementQueueFromDelayedCleanupAndMappings` (30/30 pass). - `ConsensusPrefetchingQueueDeliveryTest`: added regression tests verifying queue close/deactivation clears stale transient commit state and unblocks subsequent re-batched commits (all 15 runnable tests pass; 1 unrelated pre-existing `NoSuchMethodException` on `testSeekResetDiscardsDetachedStateAndRejectsOldCallbacksAndAcks:564` on `master`). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
