AmatyaAvadhanula commented on code in PR #13223:
URL: https://github.com/apache/druid/pull/13223#discussion_r999331401
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java:
##########
@@ -1950,6 +1950,45 @@ public Boolean
apply(SeekableStreamIndexTaskRunner.Status status)
// make sure the checkpoints are consistent with each other and with the
metadata store
verifyAndMergeCheckpoints(taskGroupsToVerify.values());
+
+ // A pause from the previous Overlord's supervisor, immediately before
leader change
+ // can lead to tasks being in a state where they are active but do not
read.
+ // If this is the first run, resume all existing active tasks to be safe
+ if (getState().isFirstRunOnly()) {
+ Map<String, ListenableFuture<Boolean>> activeTaskToResumeFutureMap = new
HashMap<>();
Review Comment:
Done
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java:
##########
@@ -1950,6 +1950,45 @@ public Boolean
apply(SeekableStreamIndexTaskRunner.Status status)
// make sure the checkpoints are consistent with each other and with the
metadata store
verifyAndMergeCheckpoints(taskGroupsToVerify.values());
+
+ // A pause from the previous Overlord's supervisor, immediately before
leader change
+ // can lead to tasks being in a state where they are active but do not
read.
+ // If this is the first run, resume all existing active tasks to be safe
+ if (getState().isFirstRunOnly()) {
+ Map<String, ListenableFuture<Boolean>> activeTaskToResumeFutureMap = new
HashMap<>();
+ for (TaskGroup taskGroup : activelyReadingTaskGroups.values()) {
+ for (String taskId : taskGroup.tasks.keySet()) {
+ activeTaskToResumeFutureMap.put(taskId,
taskClient.resumeAsync(taskId));
+ }
+ }
+
+ for (Map.Entry<String, ListenableFuture<Boolean>> entry :
activeTaskToResumeFutureMap.entrySet()) {
+ String taskId = entry.getKey();
+ ListenableFuture<Boolean> future = entry.getValue();
+ future.addListener(
+ new Runnable()
+ {
+ @Override
+ public void run()
+ {
+ try {
+ if (entry.getValue().get()) {
+ log.info("Resumed task [%s]", taskId);
+ } else {
+ log.warn("Failed to resume task [%s]", taskId);
+ killTask(taskId, "Could not resume task");
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
