jt2594838 opened a new pull request, #18693: URL: https://github.com/apache/iotdb/pull/18693
## Description When a WAL footer is missing or corrupted, metadata readers currently fail even if complete entries remain readable. Recover metadata from the readable entry prefix, recognize empty/header-only WAL files, and quarantine nonempty files with no recoverable entries as `.broken` (with a numbered suffix on collision). - Validate footer magic, lengths, and counts before deserialization. Recovery scans ignore the damaged footer and retain original encoded entry sizes, including legacy V1 entries and complete entries in a truncated uncompressed segment. - Ordinary readers reconstruct metadata in memory without replacing a file held by other readers. Startup repair writes complete entries and a new footer to a temporary file, forces and closes it, then replaces the original. Failed rewrites preserve the original file. - Keep unknown memTable IDs distinct from an empty set, close metadata channels, preserve quarantined files during startup cleanup, and avoid overwriting earlier quarantine files. V3 writer progress stored only in a damaged footer cannot be reconstructed from entry bodies; recovery uses the existing unknown/default progress values. Incomplete compressed segments cannot be decoded, so recovery retains complete readable entries before them. ## Validation - 29 tests passed across `WALFileTest`, `WALRepairWriterTest`, `WALMetaDataV3CompatibilityTest`, and `ProgressWALReaderTest`, using freshly compiled affected sources and existing local dependency artifacts in an isolated output directory. - Coverage includes V1/V2/V3, invalid footer lengths/content, LZ4, truncated uncompressed segments, empty files, quarantine collisions, failed repair preserving the original, and startup cleanup retaining quarantined files. - Spotless, Checkstyle (zero violations), and `git diff --check` passed. - The full Maven reactor test attempt on source base `34349e3ad8b` stopped before WAL tests at pre-existing `BuiltinPipePlugin` references to missing `OPC_UA_CONNECTOR` and `OPC_UA_SINK`. Upstream has since fixed those references in #18687; a full reactor run on that newer base has not yet been performed. Targeted validation is not a successful full reactor build. This PR has: - [x] been self-reviewed. - [x] added comments explaining non-obvious recovery and resource-lifecycle decisions. - [x] added unit tests and updated existing tests for the changed behavior. Key changed classes: WALMetaData, WALInputStream, WALReader, WALByteBufReader, WALFileVersion, WALWriter, WALBuffer, WALNode, WALNodeRecoverTask, WALRepairWriter. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
