maytasm commented on code in PR #14396:
URL: https://github.com/apache/druid/pull/14396#discussion_r1251333978
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java:
##########
@@ -2950,21 +2959,21 @@ private void checkTaskDuration() throws
ExecutionException, InterruptedException
}
}
-
- boolean stopTasksEarly = false;
- if (earlyStopTime != null && (earlyStopTime.isBeforeNow() ||
earlyStopTime.isEqualNow())) {
- log.info("Early stop requested - signalling tasks to complete");
-
- earlyStopTime = null;
- stopTasksEarly = true;
- }
-
-
- // if this task has run longer than the configured duration, signal all
tasks in the group to persist
- if (earliestTaskStart.plus(ioConfig.getTaskDuration()).isBeforeNow() ||
stopTasksEarly) {
+ if (stopTasksEarly) {
log.info("Task group [%d] has run for [%s]", groupId,
ioConfig.getTaskDuration());
Review Comment:
If tasks are stopped early then this log line is incorrect?
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java:
##########
@@ -2950,21 +2959,21 @@ private void checkTaskDuration() throws
ExecutionException, InterruptedException
}
}
-
- boolean stopTasksEarly = false;
- if (earlyStopTime != null && (earlyStopTime.isBeforeNow() ||
earlyStopTime.isEqualNow())) {
- log.info("Early stop requested - signalling tasks to complete");
-
- earlyStopTime = null;
- stopTasksEarly = true;
- }
-
-
- // if this task has run longer than the configured duration, signal all
tasks in the group to persist
- if (earliestTaskStart.plus(ioConfig.getTaskDuration()).isBeforeNow() ||
stopTasksEarly) {
+ if (stopTasksEarly) {
log.info("Task group [%d] has run for [%s]", groupId,
ioConfig.getTaskDuration());
futureGroupIds.add(groupId);
futures.add(checkpointTaskGroup(group, true));
+ } else if
(earliestTaskStart.plus(ioConfig.getTaskDuration()).isBeforeNow()) {
+ // if this task has run longer than the configured duration
+ // as long as the pending task groups are less than the configured
stop task count.
+ if (ioConfig.getStopTaskCount() == ioConfig.getTaskCount()
+ ||
pendingCompletionTaskGroups.values().stream().mapToInt(CopyOnWriteArrayList::size).sum()
Review Comment:
Doesn't pendingCompletionTaskGroups gets updated outside of this loop?
meaning that you would add all tasks to be checkpoint before
pendingCompletionTaskGroups gets updated?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
