[
https://issues.apache.org/jira/browse/HADOOP-18752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-18752:
------------------------------------
Release Note: The s3a connector no longer deletes directory markers by
default, which speeds up write operations, reduces iO throttling and saves
money. this can cause problems with older hadoop releases trying to write to
the same bucket. (Hadoop 3.3.0; Hadoop 3.2.x before 3.2.2, and all previous
releases). Set "fs.s3a.directory.marker.retention" to "delete" for backwards
compatibility (was: The s3a connector no longer deletes directory markers by
default; this can cause problems with older hadoop releases trying to write to
the same bucket. Set fs.s3a.directory.marker.retention to delete for backwards
compatibility)
> Change fs.s3a.directory.marker.retention to "keep"
> --------------------------------------------------
>
> Key: HADOOP-18752
> URL: https://issues.apache.org/jira/browse/HADOOP-18752
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.5
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> Change the default value of "fs.s3a.directory.marker.retention" to keep;
> update docs to match.
> maybe include with HADOOP-17802 so we don't blow up with fewer markers being
> created.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
