github-actions[bot] commented on code in PR #69255:
URL: https://github.com/apache/doris/pull/69255#discussion_r4317307004
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java:
##########
@@ -342,6 +343,7 @@ private void eraseDatabase(long currentTimeMs, int keepNum)
{
} finally {
writeUnlock();
}
+ FunctionUtil.cleanUdfCache(db);
Review Comment:
[P2] Preserve database cleanup across master failover
At this point permanent database erasure is already journaled, but
`cleanUdfCache()` retains its pending generation IDs only in the current FE's
static `AgentTaskQueue`. If that master fails while a BE is unavailable or
cleanup is pending, the successor's `replayEraseDatabase()` removes the
database and function metadata without recreating the task. After backend
recovery it has nothing to resend, leaving the per-function Python pools and
package handles resident indefinitely with default `auto_suspend=-1`. Persist
the cleanup intent through acknowledgement or reconstruct it on recovery before
discarding the last function IDs; registering retries fixes the existing
backend-outage case only while the same FE remains master.
##########
fe/fe-core/src/main/java/org/apache/doris/task/CleanUDFCacheTask.java:
##########
@@ -30,7 +30,13 @@ public class CleanUDFCacheTask extends AgentTask {
private long functionId;
public CleanUDFCacheTask(long backendId, String signature, long
functionId) {
- super(null, backendId, TTaskType.CLEAN_UDF_CACHE, -1, -1, -1, -1, -1,
-1, -1);
+ this(backendId, signature, functionId, false);
Review Comment:
[P2] Retry ordinary and global Python function cleanup too
`DropFunctionCommand.run()` still uses this three-argument constructor for
local/global DROP FUNCTION and never registers the resulting tasks in
`AgentTaskQueue`. If a BE loses its heartbeat while its Python processes remain
alive, `AgentBatchTask.run()` skips it and heartbeat recovery cannot resend the
notification. With the new per-generation pools and default `auto_suspend=-1`,
the dropped function's processes and package handles then remain indefinitely;
repeated drops accumulate pools. Unlike database erasure, these DROP paths
never call the new retry helper. Queue their ID-scoped cleanup as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
