Caideyipi opened a new pull request, #18482: URL: https://github.com/apache/iotdb/pull/18482
## Description ### Pooled TsFile parsing - Adds the per-pipe processor attribute `processor.tsfile-parser.parallelism`. - Runs built-in `DoNothingProcessor` TsFile parsing in a shared parser pool while keeping custom processors single-threaded. - Uses the existing global and per-pipe-region parser admission limits, with non-positive per-pipe-region values following the global limit. ### Ordering, lifecycle, and memory safety - Preserves ordered progress reporting across concurrently parsed TsFiles and keeps dependent events behind the parser barrier. - Allows heartbeat and progress-report control events to bypass the parser barrier. - Preserves parser state across retryable failures and handles STOP/START or task close without losing the current TsFile. - Bounds parsed-tablet output queue memory and reserves downstream tablet capacity to avoid parser/sink memory deadlocks. ### Scope This ports the TsFile parser pooling changes from the hotfix branch to master. The TsFile local sink implementation, writer, sink-specific tests, and the local-sink-based parser pool IT are intentionally excluded. ### Verification ```text .\mvnw.cmd -pl iotdb-core/datanode -am -Dtest=UnboundedBlockingPendingQueueTest,PipeProcessorSubtaskExecutorTest,PipeRawTabletInsertionEventTest,PipeMemoryManagerTest,PipeEventCollectorTest -Dsurefire.failIfNoSpecifiedTests=false test ``` Result: 28 reactor modules succeeded. Node-commons tests: 3 passed. Datanode tests: 19 passed. Checkstyle and Spotless passed. <hr> This PR has: - [x] been self-reviewed. - [x] considered concurrent read and write behavior. - [x] added or updated configuration documentation. - [x] added comments explaining non-obvious ordering, retry, and memory behavior. - [x] added or updated unit tests for the new code paths. <hr> ##### Key changed/added classes - `PipeProcessorSubtask` - `PipeEventCollector` - `PipeTsFileInsertionEvent` - `PipeRawTabletInsertionEvent` - `PipeMemoryManager` - `BlockingPendingQueue` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
