abhishekrb19 commented on code in PR #16362:
URL: https://github.com/apache/druid/pull/16362#discussion_r1585809093
##########
indexing-service/src/main/java/org/apache/druid/indexing/common/task/KillUnusedSegmentsTask.java:
##########
@@ -341,6 +342,15 @@ private NavigableMap<DateTime, List<TaskLock>>
getNonRevokedTaskLockMap(TaskActi
return taskLockMap;
}
+ @Override
+ public boolean isReady(TaskActionClient taskActionClient) throws Exception
+ {
+ if (markAsUnused) {
+ return super.isReady(taskActionClient);
+ }
+ return super.isReady(taskActionClient,
getContextValue(Tasks.TASK_LOCK_TYPE, TaskLockType.SHARED));
Review Comment:
>Having concurrent kill tasks definitely doesn't seem ideal from a resource
perspective.
Hmm. I was thinking about the plausible scenario where a user submits a
micro-batch of kill tasks rather than a large one if a datasource has too many
unused segments. That way, the user doesn't have to wait for a long-running
task to complete. Right now, if a user did that with two concurrent kill tasks
with overlapping intervals, the task that acquires the exclusive lock would do
the work and the task that follows it would essentially be a no-op when it gets
the chance to run. I'm wondering if that will still be the case?
Re idempotence: in steady state when there are no more newly added unused
segments to an interval, I'd think that the kill task should guarantee
idempotence even when `limit` and `batchSize` are supplied. Thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
