cozos commented on issue #24303: URL: https://github.com/apache/beam/issues/24303#issuecomment-1336689900
Speculative execution means: > Apache Spark has the 'speculative execution' feature to handle the slow tasks in a stage due to environment issues like slow network, disk etc. If one task is running slowly in a stage, Spark driver can launch a speculation task for it on a different host In Beam terms, I think this means that there are two SDK Harnesses writing the same bundle to the same file. > And looking at the failure, it doesn't seem to be specific to Parquet so probably affect other file-based I/O as well. Yes this doesn't seem Parquet specific so could affect other file formats. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
