[
https://issues.apache.org/jira/browse/HADOOP-16384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872527#comment-16872527
]
Steve Loughran commented on HADOOP-16384:
-----------------------------------------
Bonus treat in this: something to dump out the contents of a store as a CSV
file (example attached). This is its own CLI entry point and so can be kept out
in the corner (documented in testing?). It does currently just use the
metastore API with some casting.
> Improve resilience of S3Guard prune to inconsistent DDB tables
> --------------------------------------------------------------
>
> Key: HADOOP-16384
> URL: https://issues.apache.org/jira/browse/HADOOP-16384
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Attachments: hwdev-ireland-new.csv
>
>
> HADOOP-15183 added detection and rejection of prune updates when the store is
> inconsistent (i.e. when it tries to update an entry twice in the same
> operation, the second time with one that is inconsistent with the first)
> Now that we can detect this, we should address it. We are lucky here in that
> my DDB table is currently inconsistent: prune is failing.
> Plan
> # new test to run in the sequential phase, which does a s3guard prune against
> the bucket used in tests
> # use this to identify/debug the issue
> # replicate the problem in the ITestDDBMetastore tests
> # decide what to do in this world. Tell the user to run fsck? skip?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
