Caideyipi opened a new pull request, #18141: URL: https://github.com/apache/iotdb/pull/18141
### Background PR #18051 fixed a real self-blocking risk by releasing the TsFile parser when downstream tablet consumption hits `PipeRuntimeOutOfMemoryCriticalException`. That fallback is still necessary: with many pipes, parser-side memory and parsed tablet memory can otherwise hold each other and stop progress. However, in scenarios such as 10 pipes with device-level pattern matching, a transient consumer-side OOM currently closes the parser immediately and retries the whole TsFile event later. This turns short memory pressure into repeated local TsFile reparsing, which is very expensive. ### Changes - Reserve a TsFile parser memory permit before parsing, counted with the existing tablet/tsfile memory thresholds via `pipe_tsfile_parser_memory`, so too many TsFile parsers will not run concurrently under memory pressure. - Keep the current parser and retry the current parsed tablet locally for a bounded short window when the consumer reports OOM. - Fall back to releasing the parser and retrying the whole TsFile event when hard memory pressure is detected or the retry budget/time is exhausted. - Release the parser memory reservation on normal close and reference-resource finalization. ### Tests - `mvn -pl iotdb-core/datanode -Dtest=TsFileInsertionEventParserTest#testConsumeTabletInsertionEventsWithRetryReleasesParserOnOutOfMemory+testConsumeTabletInsertionEventsWithRetryKeepsParserForTransientOutOfMemory -DforkCount=0 test` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
