yujun777 commented on code in PR #22889:
URL: https://github.com/apache/doris/pull/22889#discussion_r1434810351
##########
fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadManager.java:
##########
@@ -679,19 +681,35 @@ public void processTimeoutTasks() {
// Remove old routine load jobs from idToRoutineLoadJob
// This function is called periodically.
- // Cancelled and stopped job will be remove after
Configure.label_keep_max_second seconds
+ // Cancelled and stopped job will be removed after
Configure.label_keep_max_second seconds
public void cleanOldRoutineLoadJobs() {
LOG.debug("begin to clean old routine load jobs ");
+ clearRoutineLoadJobIf(RoutineLoadJob::isExpired, -1);
+ }
+
+ /**
+ * Remove finished routine load jobs from idToRoutineLoadJob
+ * This function is called periodically if Config.label_num_threshold is
set.
+ * Cancelled and stopped job will be removed.
+ */
+ public void cleanOverLimitRoutineLoadJobs() {
+ if (idToRoutineLoadJob.size() <= Config.label_num_threshold) {
+ return;
+ }
+ LOG.debug("begin to clean routine load jobs");
+ clearRoutineLoadJobIf(RoutineLoadJob::isFinal,
Config.label_num_threshold);
Review Comment:
Is it better to remove oldest jobs when exceed limit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
