steveloughran commented on a change in pull request #2530:
URL: https://github.com/apache/hadoop/pull/2530#discussion_r555075093
##########
File path:
hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
##########
@@ -1873,11 +1873,9 @@
<property>
<name>fs.s3a.committer.magic.enabled</name>
- <value>false</value>
+ <value>true</value>
Review comment:
I' will add to release notes. This *does not* enable the committer. All
it does is say "this store has the consistency needed for the committer".
Now, we could do a bit more in a separate patch as there's a bit more in
settings I'd like to change (we ship in cloudera) where the fs.s3a.buffer.dir
is set to something like `${env.LOCAL_DIR:${hadoop.tmp.dir}}/s3a` so that
incomplete buffered writes (staging committer and blocks for normal files/magic
uploads) are always cleaned up. I could make the core-default settings for this
committer different
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
