Caideyipi opened a new pull request, #18818: URL: https://github.com/apache/iotdb/pull/18818
## Description Continuous subscription WAL replay could permanently skip unreadable retained files while `subscription_consensus_wal_gap` remained zero. The detailed warning only ran when the iterator was exhausted, and clients received a noncritical error containing only a file count. This change makes file skips observable and diagnosable: - Add queue/region metrics `subscription_consensus_wal_file_gap` and `subscription_consensus_wal_file_gap_entries`, plus queue diagnostics. Counters accumulate across iterator resets. File counts measure skip occurrences; entry counts include only known skipped WAL entries/fragments, including discarded incomplete request fragments. An unreadable footer leaves its entry count explicitly unknown. - Log each permanent skip immediately with its absolute filename, version ID, file-level searchIndex bounds, entry offsets and exception. Send a critical poll error with queue/region context and up to eight recent file details; the server log contains every skipped file. - Retry EOF at the original file/entry offset while preserving incomplete requests. Only a repeated EOF at the same offset in an unchanged sealed retained file confirms truncation. Near-live EOF and ordinary I/O failures wait for another replay attempt. Preserve the existing escape path for entries above the supported WAL size limit, with an explicit gap signal. - Preserve a critical error from parallel consumer poll tasks even when sibling topics return data. Healthy messages are returned once, and the next poll raises the saved critical exception instead of permanently reducing it to a warning. `subscription_consensus_wal_gap` retains its local searchIndex unit. Monitoring and regression checks must require both it and `subscription_consensus_wal_file_gap` to be zero; a zero known-entry count alone does not exclude a file gap. Filename searchIndex bounds describe the whole file and are not an inferred row-loss count. ## Validation - Full English reactor build through `test`: 88 tests in eight targeted subscription/WAL classes, zero failures/errors, one optional performance test skipped. - Full Chinese reactor `test-compile -DskipTests -P with-zh-locale`: passed (53 modules). - Spotless applied to DataNode, Node Commons and Subscription Client; new English/Chinese message keys and format specifiers match. This PR has: - [x] been self-reviewed. - [x] added comments explaining the retry, counting and error-delivery behavior. - [x] added or updated unit tests for transient/sealed/live EOF, preservation of incomplete fragments, immediate file warnings, bounded diagnostics, cumulative/region-scoped gauges, and delivery of a critical error after healthy parallel poll results. The supplied regression logs confirm six noncritical skip errors, zero detailed iterator warnings and 5,400 zero-valued WAL-gap samples. Those logs cannot identify the skipped files or establish a row-loss count; the original high-load benchmark has not been rerun. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
