Caideyipi commented on PR #19243: URL: https://github.com/apache/iotdb/pull/19243#issuecomment-6818664360
Initial independent self-review: one round completed. Found and repaired one behavioral issue. - **[P1] Processed Tablet descendants could release the TsFile ordering gate too early.** The release fallback at [PipeTsFileInsertionEvent.java:716-723](https://github.com/apache/iotdb/blob/e6a74a43ad79ae84c63b0837ee3e5f78ad403e33/iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/PipeTsFileInsertionEvent.java#L716-L723) can see zero generated events after a processor calls `processRowByRow()`: [PipeEventCollector.java:287-294](https://github.com/apache/iotdb/blob/e6a74a43ad79ae84c63b0837ee3e5f78ad403e33/iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/agent/task/connection/PipeEventCollector.java#L287-L294) and [PipeRawTabletInsertionEvent.java:138-150](https://github.com/apache/iotdb/blob/e6a74a43ad79ae84c63b0837ee3e5f78ad403e33/iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tablet/PipeRawTabletInsertionEvent.java#L138-L150) only recognized immediate TsFile parents. The processed output has a raw Tablet parent, so r eleasing the original could admit the next file before that output committed or was discarded. - **Fix:** retain the original TsFile ancestor across raw Tablet transformations, register each collected descendant with it, and send commit/discard outcomes to that ancestor. See [ancestor and commit tracking](https://github.com/apache/iotdb/blob/4d311c88a4691425e4017a96877138c9365c3d23/iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tablet/PipeRawTabletInsertionEvent.java#L119-L155), [discard tracking](https://github.com/apache/iotdb/blob/4d311c88a4691425e4017a96877138c9365c3d23/iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tablet/PipeRawTabletInsertionEvent.java#L370-L378), and [collector registration](https://github.com/apache/iotdb/blob/4d311c88a4691425e4017a96877138c9365c3d23/iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/agent/task/connection/PipeEventCollector.java#L287-L295). - **Regression coverage added:** [PipeRealtimeExtractTest.java:1654-1786](https://github.com/apache/iotdb/blob/4d311c88a4691425e4017a96877138c9365c3d23/iotdb-core/datanode/src/test/java/org/apache/iotdb/db/pipe/source/PipeRealtimeExtractTest.java#L1654-L1786) writes a real tree TsFile, parses it, processes rows, and publishes through the real collector. Assertions require both outputs to finish, reject duplicate completion after repeated cleanup, and keep an original with a committer key blocked behind an earlier commit ID. The two existing new table-file tests exercise the real collector for both original-release/Tablet-commit orders; the modified generation-boundary test covers unfinished generation. Validation performed: static inspection of the full reviewed diff and relevant lifecycle/commit paths, repair signature/call-site checks, `git diff --check`, and scoped `mvn spotless:apply -P with-integration-tests -pl iotdb-core/datanode` (all three modified Java files processed). **Compilation and tests were not run, as requested.** All five newly added PR tests remain unrun locally. CI was neither awaited nor polled; no CI result is claimed. - Reviewed SHA: `e6a74a43ad79ae84c63b0837ee3e5f78ad403e33` - Final local SHA: `4d311c88a4691425e4017a96877138c9365c3d23` - Confirmed remote source-branch SHA: `4d311c88a4691425e4017a96877138c9365c3d23` - Confirmed PR head SHA: `4d311c88a4691425e4017a96877138c9365c3d23` The source branch was checked for concurrent updates before the fast-forward update, with an expected-head lease protecting the push. Both remote heads were confirmed afterward. The PR remains **Draft** as requested. Because this round found a behavioral issue, a fresh-session independent re-review is required; this round is not an issue-free approval. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
