luoyuxia commented on PR #21257: URL: https://github.com/apache/flink/pull/21257#issuecomment-1367081179
> After discussion with @luoyuxia offline, we would not support speculative execution in the this version. And we would continue on this work after [FLIP-281+Sink+Supports+Speculative+Execution+For+Batch+Job](https://cwiki.apache.org/confluence/display/FLINK/FLIP-281+Sink+Supports+Speculative+Execution+For+Batch+Job) is done. Please fire a JIRA to issue the future work. Yes. We can continue this after [FLIP-281+Sink+Supports+Speculative+Execution+For+Batch+Job](https://cwiki.apache.org/confluence/display/FLINK/FLIP-281+Sink+Supports+Speculative+Execution+For+Batch+Job). But I don't think we need any issue the future work. For current design, among these four operatos, the only concern is the `BatchPartitionCommitterSink` in Speculative Execution. In Speculative Execution, it may cause to commit files for concurrently which is problematic. But as FLIP-281 said, if sink function doesn't implement `SupportsConcurrentExecutionAttempts`, speculative execution will be disabled. So, if the FLIP-281, we don't need to do any extra thing for the `BatchPartitionCommitterSink` will disable speculative execution by default. I may miss something, please correct me if I'm wrong. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
