[
https://issues.apache.org/jira/browse/HADOOP-12269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14644403#comment-14644403
]
Thomas Demoor commented on HADOOP-12269:
----------------------------------------
Setting defaults is difficult as the "correct value depends greatly on the
bandwidth between your Hadoop cluster and the S3(-compatible) backend and the
reason you use multipartupload: limit size of "unit that gets lost" on a bad
connection (WAN scenario), starting upload before all data is in
(s3a.fast.upload=true), ...
I'm setting it back to 2GB (although that's certainly a lot too high for WAN).
Adjusting the defaults merits its own JIRA as the other s3a defaults are also,
mmh, not fantastic.
> Update aws-sdk dependency to 1.10.6
> -----------------------------------
>
> Key: HADOOP-12269
> URL: https://issues.apache.org/jira/browse/HADOOP-12269
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Thomas Demoor
> Assignee: Thomas Demoor
> Attachments: HADOOP-12269-001.patch
>
>
> This was originally part of HADOOP-11684, pulling out to this separate
> subtask as requested by [[email protected]]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
