[
https://issues.apache.org/jira/browse/HADOOP-16384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16880536#comment-16880536
]
Steve Loughran edited comment on HADOOP-16384 at 7/8/19 4:55 PM:
-----------------------------------------------------------------
Notable that some failures all come from the listings of entries created in the
same "inconsistent listing" test
{code}
"type" "deleted" "path" "is_auth_dir" "is_empty_dir" "len"
"updated" "updated_s" "last_modified" "last_modified_s" "etag"
"version"
"s3a://hwdev-steve-ireland-new/fork-0009/test/a/b/dir3-DELAY_LISTING_ME"
"dir" "false" "" "UNKNOWN" 0 "" 1562604657792
"Mon Jul 08 17:50:57 BST 2019" "" ""
"s3a://hwdev-steve-ireland-new/fork-0009/test/rolling/1" "dir" "false"
"" "UNKNOWN" 0 "" 1562604657792 "Mon Jul 08
17:50:57 BST 2019" "" ""
"s3a://hwdev-steve-ireland-new/test/DELAY_LISTING_ME" "dir" "false" ""
"UNKNOWN" 0 "" 1562604657792 "Mon Jul 08 17:50:57
BST 2019" "" ""
{code}
was (Author: [email protected]):
Notable that some failures all come from the listings of entries created in the
same "inconsistent listing" test
{code}
"type" "deleted" "path" "is_auth_dir" "is_empty_dir" "len"
"updated" "updated_s" "last_modified" "last_modified_s" "etag"
"version"
"s3a://hwdev-steve-ireland-new/fork-0009/test/a/b/dir3-DELAY_LISTING_ME"
"dir" "false" "" "UNKNOWN" 0 "" 1562604657792
"Mon Jul 08 17:50:57 BST 2019" "" ""
"s3a://hwdev-steve-ireland-new/fork-0009/test/rolling/1" "dir" "false"
"" "UNKNOWN" 0 "" 1562604657792 "Mon Jul 08
17:50:57 BST 2019" "" ""
"s3a://hwdev-steve-ireland-new/test/DELAY_LISTING_ME" "dir" "false" ""
"UNKNOWN" 0 "" 1562604657792 "Mon Jul 08 17:50:57
BST 2019" "" ""
{code}
> ITestS3AContractRootDir failing: inconsistent DDB tables
> --------------------------------------------------------
>
> Key: HADOOP-16384
> URL: https://issues.apache.org/jira/browse/HADOOP-16384
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Attachments: hwdev-ireland-new.csv
>
>
> HADOOP-15183 added detection and rejection of prune updates when the store is
> inconsistent (i.e. when it tries to update an entry twice in the same
> operation, the second time with one that is inconsistent with the first)
> Now that we can detect this, we should address it. We are lucky here in that
> my DDB table is currently inconsistent: prune is failing.
> Plan
> # new test to run in the sequential phase, which does a s3guard prune against
> the bucket used in tests
> # use this to identify/debug the issue
> # replicate the problem in the ITestDDBMetastore tests
> # decide what to do in this world. Tell the user to run fsck? skip?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
