steveloughran commented on pull request #30714: URL: https://github.com/apache/spark/pull/30714#issuecomment-769849804
I'll update the docs. Also going to cut back on the warnings now that AWS S3 is consistent (And all third party ones were consistent out the box too). This is a good thing for apps like spark (much easier to do workflows across applications), but you still can't safely use the rename committer there as dir rename is non-atomic. Fortunately, commit-by-rename is so slow people have a reason to switch to a better one, either the S3A ones or that from EMR :) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
