[
https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-13345:
------------------------------------
Affects Version/s: 2.8.1
Release Note:
S3Guard (pronounced see-guard) is a new feature for the S3A connector to Amazon
S3, which uses DynamoDB for a high performance and consistent metadata
repository. Essentially: S3Guard caches directory information, so your S3A
clients get faster lookups and resilience to inconsistency between S3 list
operations and the status of objects. When files are created, with S3Guard,
they'll always be found.
S3Guard does not address update consistency: if a file is updated, while the
directory information will be updated, calling open() on the path may still
return the old data. Similarly, deleted objects may also potentially be opened.
Please consult the S3Guard documentation in the Amazon S3 section of our
documentation.
Note: part of this update includes moving to a new version of the AWS SDK 1.11,
one which includes the Dynamo DB client and its a shaded version of Jackson 2.
The large aws-sdk-bundle JAR is needed to use the S3A client with or without
S3Guard enabled. The good news: because Jackson is shaded, there will be no
conflict between any Jackson version used in your application and that which
the AWS SDK needs.
> S3Guard: Improved Consistency for S3A
> -------------------------------------
>
> Key: HADOOP-13345
> URL: https://issues.apache.org/jira/browse/HADOOP-13345
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs/s3
> Affects Versions: 2.8.1
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HADOOP-13345.prototype1.patch, s3c.001.patch,
> S3C-ConsistentListingonS3-Design.pdf, S3GuardImprovedConsistencyforS3A.pdf,
> S3GuardImprovedConsistencyforS3AV2.pdf
>
>
> This issue proposes S3Guard, a new feature of S3A, to provide an option for a
> stronger consistency model than what is currently offered. The solution
> coordinates with a strongly consistent external store to resolve
> inconsistencies caused by the S3 eventual consistency model.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
