[
https://issues.apache.org/jira/browse/HADOOP-17403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17240956#comment-17240956
]
Steve Loughran commented on HADOOP-17403:
-----------------------------------------
{code}
[ERROR]
testRenameDirFailsInDelete[bulk-delete=true](org.apache.hadoop.fs.s3a.impl.ITestPartialRenamesDeletes)
Time elapsed: 13.118 s <<< FAILURE!
java.lang.AssertionError:
[files which could not be deleted]
Expecting:
<[s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/dir-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-008,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-009,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-004,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/dir-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/dir-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-005,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/dir-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-006,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-009,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/dir-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-005,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-007,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-008,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-008,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-009,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-004,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/dir-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-008,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/dir-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/dir-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-007,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-005,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-009,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/dir-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/dir-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-005,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/dir-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/dir-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/dir-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-004,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-008,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-009,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-007,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-006,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-005,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/dir-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/dir-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-007,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/dir-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-004,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-006,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-004,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-007,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-006,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-006,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-002]>
to contain exactly in any order:
<[s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-004,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-005,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-006,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-007,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-008,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/file-009,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-004,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-005,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-006,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-007,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-008,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/file-009,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-004,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-005,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-006,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-007,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-008,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/file-009,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-004,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-005,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-006,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-007,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-008,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/file-009,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-004,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-005,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-006,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-007,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-008,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/file-009,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/dir-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/dir-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/dir-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-000/dir-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/dir-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/dir-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/dir-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-001/dir-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/dir-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/dir-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/dir-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002/dir-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/dir-000,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/dir-001,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/dir-002,
s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-003/dir-003]>
but could not find the following elements:
<[s3a://stevel-london/test/testRenameDirFailsInDelete[bulk-delete=true]-multi-1606758234.617/readonlyDir/dir-002]>
at
org.apache.hadoop.fs.s3a.impl.ITestPartialRenamesDeletes.testRenameDirFailsInDelete(ITestPartialRenamesDeletes.java:522)
{code}
> S3A ITestPartialRenamesDeletes failure: missing directory marker
> ----------------------------------------------------------------
>
> Key: HADOOP-17403
> URL: https://issues.apache.org/jira/browse/HADOOP-17403
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.1
> Reporter: Steve Loughran
> Priority: Minor
>
> Seemingly transient failure of the test ITestPartialRenamesDeletes with the
> latest HADOOP-17244 changes in: an expected directory marker was not found.
> Test run was (unintentionally) sequential, markers=delete, s3guard on
> {code}
> -Dmarkers=delete -Ds3guard -Ddynamo -Dscale
> {code}
> Hasn't come back since.
> The bucket's retention policy was authoritative, but no dirs were declared as
> such
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
