VGalaxies opened a new pull request, #19340:
URL: https://github.com/apache/iotdb/pull/19340

   ## Problem
   
   When a `ConsensusPrefetchingQueue` closes, deactivates, or is replaced by a 
new queue instance (for example after consumer disconnection, failover, or 
restart), transient in-flight tracking state inside 
`ConsensusSubscriptionCommitState` (`outstandingKeys`, 
`outstandingKeysByWriter`, `committedPendingKeysByWriter`, and uncommitted 
entries in `recentlyCommittedKeys`) remained intact across queue lifecycles.
   
   When a replacement queue replayed from the committed frontier and re-batched 
WAL entries into different batch boundaries:
   1. Old uncommitted `outstandingKeys` from the previous queue lifecycle could 
never be acknowledged again, pinning the contiguous commit watermark 
(`committedWriterPositions`) and freezing commit progress.
   2. Uncommitted out-of-order ACKs retained in `recentlyCommittedKeys` could 
interfere with newly mapped replayed batches.
   3. Delayed async cleanup, mapping, or commit calls from a superseded queue 
could corrupt the live transient state of the replacement queue.
   
   ## Fix
   
   - Add transient owner fencing (`transientOwner`) and transient in-flight 
state lifecycle methods (`claimTransientInFlightState`, 
`clearTransientInFlightStateIfOwner`, `clearTransientInFlightState`) on 
`ConsensusSubscriptionCommitState` / `ConsensusSubscriptionCommitManager`.
   - Prune uncommitted entries (`> committedKey`) from `recentlyCommittedKeys` 
when clearing transient in-flight state while retaining already-committed keys 
for idempotent re-commit.
   - Enforce `transientOwner` fencing atomically inside 
`ConsensusSubscriptionCommitState`'s monitor (`synchronized (this)`) for 
`recordMapping`, `commit`, and `commitWithoutOutstanding`, preventing TOCTOU 
races between owner validation and state mutation.
   - Claim transient ownership in `ConsensusPrefetchingQueue` during 
initialization and seek reset, and clear transient state for the current owner 
on close and deactivation.
   
   ## Tests
   
   - `ConsensusSubscriptionCommitStateTest`: added 
`testClearTransientInFlightStateUnblocksRebatchedReplayAfterRestart` and 
`testTransientOwnerFencingProtectsReplacementQueueFromDelayedCleanupAndMappings`
 (30/30 pass).
   - `ConsensusPrefetchingQueueDeliveryTest`: added regression tests verifying 
queue close/deactivation clears stale transient commit state and unblocks 
subsequent re-batched commits (all 15 runnable tests pass; 1 unrelated 
pre-existing `NoSuchMethodException` on 
`testSeekResetDiscardsDetachedStateAndRejectsOldCallbacksAndAcks:564` on 
`master`).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to