hongzhi-gao opened a new pull request, #18872: URL: https://github.com/apache/iotdb/pull/18872
## Description A TsFileProcessor can remain open after its working memtable is detached for flush. Tree and table deletions currently dereference that null memtable while logging to WAL. Handling this state must also preserve deletion replay and prevent a deletion registered during flush completion from being acknowledged without being persisted. ### Deletion logging and flush ordering - Associate a deletion with the working memtable, or the last pending flushing memtable when no working memtable exists. Use the existing `MEMTABLE_NOT_EXIST` sentinel only when neither remains, while retaining the WAL entry for IoTConsensus replication. - Hold the existing flush-query read lock through ID selection and WAL enqueue, before the selected memtable can be released and checkpointed. - Persist queued modifications and release the flushed memtable under one write-lock critical section. This closes the interval where a deletion could be registered after the modifications scan and then lost during release. Preserve the existing deque-monitor-to-write-lock order. ### Shared-file WAL recovery Map every unflushed memtable ID in a TsFile to the same recovery performer. Deduplicate performer lifecycle calls so the shared file is started, finalized, and closed once. The WAL and RPC formats remain unchanged. ### Validation ```bash mvn spotless:apply -pl iotdb-core/datanode mvn spotless:check -o -pl iotdb-core/datanode mvn test -o -pl iotdb-core/datanode -am -Dtest=TsFileProcessorTest,WALRecoverManagerTest,UnsealedTsFileRecoverPerformerTest -Dsurefire.failIfNoSpecifiedTests=false git diff --check ``` All 36 tests passed, with no failures, errors, or skipped tests. The new tree/table WAL tests reproduce the NPE on the original implementation. The synchronized flush-completion regression fails before the locking fix because the persisted modifications are empty, and passes after the fix. The recovery regression uses real checkpoint/WAL files and checks both deleted and retained values after recovery. English and Chinese reactor test compilation compiled DataNode and the preceding modules successfully: ```bash mvn test-compile -o -DskipTests mvn test-compile -o -DskipTests -P with-zh-locale ``` Both offline runs initially stopped at `distribution` due to uncached snapshot ZIP dependencies. Fetching those dependencies and resuming with `mvn test-compile -DskipTests -rf :iotdb-distribution` passed for distribution and all remaining modules. Cluster tests, process-crash fault injection, and disk-I/O fault injection were not run. <hr> This PR has: - [x] been self-reviewed, including concurrent deletion/flush and recovery ordering. - [x] added comments explaining the intent and lock ordering. - [x] added regression tests for deletion WAL logging, flush completion, and shared-file recovery. <hr> ##### Key changed classes - `TsFileProcessor` - `WALNodeRecoverTask` - `TsFileProcessorTest` - `WALRecoverManagerTest` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
