HI all, The default OutputCommitter used by RDD, which is FileOutputCommitter, seems to require moving files at the commit step, which is not a constant operation in S3, as discussed in http://mail-archives.apache.org/mod_mbox/spark-user/201410.mbox/%3c543e33fa.2000...@entropy.be%3E. People seem to develop their own NullOutputCommitter implementation or use DirectFileOutputCommitter (as mentioned in SPARK-3595<https://issues.apache.org/jira/browse/SPARK-3595>), but I wanted to check if there is a de facto standard, publicly available OutputCommitter to use for S3 in conjunction with Spark.
Thanks, Mingyu
