azagrebin commented on issue #7757: [FLINK-11630] Triggers the termination of all running Tasks when shutting down TaskExecutor URL: https://github.com/apache/flink/pull/7757#issuecomment-496118950 @kisimple I agree with @tillrohrmann , it looks fragile to rely on `taskSlotTable` for querying incomplete tasks. Depending on when the `Task.notifyFinalState` is called, it can happen that some resources are still in use in Task and we release network concurrently. Although, it is not the case now but we can miss it in future if `Task.notifyFinalState` is moved somewhere else. Instead of relying on `taskSlotTable` we can introduce another container (e.g. `incompleteTasks`) to keep undone tasks in `TaskExecutor`. The tasks can be added to `incompleteTasks` in `TaskExecutor.submitTask` when the task is created. We can also immediately add a callback to its close future with the task removing itself from `incompleteTasks` when is done. Finally we can use `incompleteTasks` instead of `taskSlotTable` in `TaskExecutor.failTasks` to query complete futures and wait for them. Apart from other @tillrohrmann's comments, it seems we can simplify more and do all `stopTaskExecutorServices` after all tasks are completed.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
