[
https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15453539#comment-15453539
]
Lei (Eddy) Xu commented on HADOOP-13345:
----------------------------------------
Hi, [~fabbri] thanks for these great suggestions here.
One question here is:
* Can we consider {{fullycache.directories == true iff
metadatastore.allow.authoritative == true}}? If we combine them together, case
2 of {{fullycache.directories}} should not happen.
bq. as the MetadataStore will always return results marked as non-authoritative.
If we have this flag, we might not need to mark results as well.
So I think the code like following can make the things simpler:
{code}
List<PathMetadata> subFiles = metadataStore.get(path);
if (metadataStore.isAuthoritive()) {
List<PathMetadata> s3Files = s3.listDir(path);
// merge subfile and s3Files...
}
{code}
What do you think?
> S3Guard: Improved Consistency for S3A
> -------------------------------------
>
> Key: HADOOP-13345
> URL: https://issues.apache.org/jira/browse/HADOOP-13345
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs/s3
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HADOOP-13345.prototype1.patch,
> S3C-ConsistentListingonS3-Design.pdf, S3GuardImprovedConsistencyforS3A.pdf,
> S3GuardImprovedConsistencyforS3AV2.pdf, s3c.001.patch
>
>
> This issue proposes S3Guard, a new feature of S3A, to provide an option for a
> stronger consistency model than what is currently offered. The solution
> coordinates with a strongly consistent external store to resolve
> inconsistencies caused by the S3 eventual consistency model.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
