Caideyipi opened a new pull request, #18805: URL: https://github.com/apache/iotdb/pull/18805
## Description Resubmit #18741 after its revert in #18786, on current `master` and with #18785's linger-batch flush retained. ### Why a subscription can no longer find WAL The incident showed the local replay cursor expecting search index 57 while the next visible WAL request was 113. The older retention calculation used committed subscription progress alone. A different consumer or replica could advance that committed progress while this queue still needed an earlier local search index; WAL cleanup could then remove a segment before this queue replayed it. #18769 already changed the retention bound to the earlier of committed progress and this queue's local replay cursor. This PR restores the replay-side guard so any remaining missing range is detected instead of silently skipped. An apparent gap can also be temporary when the active WAL has not become readable yet. On the first gap, the queue rolls the active WAL and recreates its iterator at the expected index. If the range is still unavailable on retry, it keeps the cursor at the missing index, stops delivery and real-time admission, marks progress as stalled, and returns a critical error to the consumer. ### Seek and #18785 compatibility - Preserve #18785's flush of an existing linger batch before a pending WAL gap retry. - Bind gap validation to the seek generation. A result from before a seek is treated as stale; applying a seek clears the old gap and stalled state. - Resolve `seekToBeginning` to the earliest replayable WAL entry. For seeks to the tail, continue after the last existing search index. This prevents an intentional seek from being reported as a missing WAL range. - Keep the last unread local WAL file protected by the retention boundary already present on `master`. ### Verification - `ConsensusPrefetchingQueueTest`: 44 tests passed (`surefire:test -DforkCount=0`), including WAL gap retry, critical failure, seek recovery, and the at-end cursor boundary. - Spotless and Checkstyle passed for DataNode. - The full English reactor passed through DataNode and later modules, then stopped in `distribution`: its snapshot ZIP dependencies could not be downloaded from the internal Nexus repository (HTTP 500). - Chinese locale reactor test compilation through DataNode and its upstream modules passed (`mvn test-compile -pl iotdb-core/datanode -am -P with-zh-locale -DskipTests`). This PR has: - [x] been self-reviewed, including the seek/gap interaction. - [x] added comments for the WAL retention and seek boundaries. - [x] added or updated unit tests for the changed paths. ##### Key changed/added classes - `ConsensusPrefetchingQueue`: WAL replay continuity, seek generation handling, and seek cursor normalization. - `ConsensusPrefetchingQueueTest`: gap retry, stalled delivery, seek recovery, and tail cursor regression coverage. - `DataNodePipeMessages` (en/zh): localized gap retry and critical error messages. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
