Caideyipi opened a new pull request, #18912:
URL: https://github.com/apache/iotdb/pull/18912

   ## Description
   
   After a transient realtime admission failure, subscription WAL gap recovery 
can replay requests that have already been admitted into the pending queue. For 
example, rejecting index 1 and then admitting indexes 2–8 causes all eight 
requests to be delivered through WAL, while their pending copies are discarded 
as duplicates. This prevents the recovered realtime path from supplying those 
requests.
   
   Stop gap replay at the next pending index and resume direct pending 
delivery. If a sparse WAL jumps to or past that boundary, retain its buffered 
request and report only the missing prefix. Replicated requests without local 
search indexes before the boundary are still replayed. Existing scan budgets, 
memory limits, ordering, and commit handling continue to apply.
   
   ## Validation
   
   - Confirmed both admission-recovery regressions fail before the fix: WAL 
accepts 8 and 2 requests respectively instead of the single missing request.
   - Added real-WAL coverage for consensus-memory rejection, ACK-driven 
recovery under materialization memory pressure, a missing WAL prefix, and 
replicated writer entries before the pending boundary. Assertions cover 
delivery order, source counters, committed progress, and released memory.
   - Made the existing mixed pending/WAL redelivery fixture independent of 
cold-start scan deadlines; confirmed its original single-batch assertion also 
fails on unmodified master when the scan yields.
   - Full English reactor: `mvn -nsu test 
-Dtest=ConsensusPrefetchingQueue*Test,ProgressWALIteratorTest 
-Dsurefire.failIfNoSpecifiedTests=false -Ddevelocity.off=true` succeeds across 
all 53 modules; 111 tests pass and one opt-in performance test is skipped.
   - Full Chinese reactor: `mvn -nsu test-compile -Pwith-zh-locale -DskipTests 
-Ddevelocity.off=true` succeeds across all 53 modules.
   - `mvn spotless:apply -pl iotdb-core/datanode` and `git diff --check` pass.
   
   The original pressure workload has not been rerun in a cluster.
   
   This PR has:
   
   - [x] been self-reviewed.
   - [x] added comments explaining the replay boundary and sparse-WAL handling.
   - [x] added unit tests covering the changed behavior.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to