tillrohrmann commented on issue #6684: [FLINK-10205] Batch Job: InputSplit Fault tolerant for DataSource… URL: https://github.com/apache/flink/pull/6684#issuecomment-434628107 Thanks for the explanation @isunjin. Now I understand that you don't think that it is strictly required to let a failed task process exactly the same `InputSplits` and that it is just a side effect of the current implementation. So in the end you've implemented it this way, because the `InputSplitAssigner` does not support returning `InputSplits`. Maybe that is something we should change. We could, for example, add a new interface which needs to be implemented by an `InputSplitAssigner` to support fine grained recovery. Otherwise, such a failure will result into a global failover. My concern is that by storing `InputSplits` in the `Executions` that we are mixing a bit of concerns. For example, assume that we have three tasks failing and we also lost a slot. Then we could only restart two of these tasks and need to distribute the slots of the third `Execution` among the newly started `Executions`. It would be much easier to simply return all slots to the `InputSplitAssigner` and let the newly started `Executions` pull from there.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
