github-actions[bot] commented on code in PR #66473:
URL: https://github.com/apache/doris/pull/66473#discussion_r3780581260
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/StatementContext.java:
##########
@@ -910,6 +917,7 @@ protected void finalize() throws Throwable {
@Override
public void close() {
+ clearExternalScanTasks();
Review Comment:
[P1] Release cached tasks from direct job executions
`StatementContext.close()` (or an explicit clear) is now required to release
this cache, but the fixed-pool `InsertTask` and `MTMVTask` paths never do
either. They install a `ConnectContext` in thread-local storage, execute their
Nereids commands directly, and later only null task-owned fields;
`TaskProcessor` has no finally cleanup. After an external source is planned, an
idle worker can therefore keep `ConnectContext -> StatementContext ->
ExternalScanTaskCache` reachable after task completion, including up to 10,000
Hive/Hudi tasks and the independent 16 MiB Iceberg/Paimon budgets per worker.
Please put cleanup on every execution surface, or close the statement context
and remove/restore the thread-local context from these job paths, with an
external INSERT/MTMV completion test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
