steveloughran commented on issue #992: HADOOP-16357 TeraSort Job failing on S3 DirectoryStagingCommitter: destination path exists URL: https://github.com/apache/hadoop/pull/992#issuecomment-510107381 we're changing the default mode to == that of FileOutputCommitter, namely "it's ok if the dest exists" Spark will refuse to execute if the dest exists, so to you use any of the conflict options you actually need to permit overwrite -so you won't change behaviour without noticing What this does do is make the committer more consistent with the existing one
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
