[
https://issues.apache.org/jira/browse/HADOOP-13713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695908#comment-15695908
]
Steve Loughran commented on HADOOP-13713:
-----------------------------------------
Still seeing this intermittently
{code}
Failed tests:
ITestS3AContractRootDir>AbstractContractRootDirectoryTest.testRmEmptyRootDirNonRecursive:95->Assert.fail:88
After 1 attempts: listing after rm /* not empty
final [00] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-2;
isDirectory=true; modification_time=0; access_time=0; owner=stevel;
group=stevel; permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
deleted [00] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/file.txt;
isDirectory=false; length=0; replication=1; blocksize=33554432;
modification_time=1480011550000; access_time=0; owner=stevel; group=stevel;
permission=rw-rw-rw-; isSymlink=false} isEmptyDirectory=false
[01] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/Users; isDirectory=true;
modification_time=0; access_time=0; owner=stevel; group=stevel;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
[02] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-1; isDirectory=true;
modification_time=0; access_time=0; owner=stevel; group=stevel;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
[03] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-2; isDirectory=true;
modification_time=0; access_time=0; owner=stevel; group=stevel;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
[04] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-3; isDirectory=true;
modification_time=0; access_time=0; owner=stevel; group=stevel;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
[05] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-4; isDirectory=true;
modification_time=0; access_time=0; owner=stevel; group=stevel;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
[06] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-5; isDirectory=true;
modification_time=0; access_time=0; owner=stevel; group=stevel;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
[07] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/path; isDirectory=true;
modification_time=0; access_time=0; owner=stevel; group=stevel;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
[08] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/test; isDirectory=true;
modification_time=0; access_time=0; owner=stevel; group=stevel;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
original [00] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/file.txt;
isDirectory=false; length=0; replication=1; blocksize=33554432;
modification_time=1480011550000; access_time=0; owner=stevel; group=stevel;
permission=rw-rw-rw-; isSymlink=false} isEmptyDirectory=false
[01] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/Users; isDirectory=true;
modification_time=0; access_time=0; owner=stevel; group=stevel;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
[02] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-1; isDirectory=true;
modification_time=0; access_time=0; owner=stevel; group=stevel;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
[03] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-2; isDirectory=true;
modification_time=0; access_time=0; owner=stevel; group=stevel;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
[04] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-3; isDirectory=true;
modification_time=0; access_time=0; owner=stevel; group=stevel;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
[05] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-4; isDirectory=true;
modification_time=0; access_time=0; owner=stevel; group=stevel;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
[06] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-5; isDirectory=true;
modification_time=0; access_time=0; owner=stevel; group=stevel;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
[07] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/path; isDirectory=true;
modification_time=0; access_time=0; owner=stevel; group=stevel;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
[08] S3AFileStatus{path=s3a://hwdev-steve-ireland-new/test; isDirectory=true;
modification_time=0; access_time=0; owner=stevel; group=stevel;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
{code}
What I think may be happening is that we are spinning waiting for the delete to
complete, which doesn't. But there's no attempt to retry the rm /; so if that
first listing wasn't complete, it'll never get rid of later arrivals in the
listing. Fix: include the RM in the loop
> ITestS3AContractRootDir.testRmEmptyRootDirNonRecursive failing intermittently
> -----------------------------------------------------------------------------
>
> Key: HADOOP-13713
> URL: https://issues.apache.org/jira/browse/HADOOP-13713
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.8.0
> Environment: s3 ireland
> Reporter: Steve Loughran
> Assignee: Steve Loughran
>
> intermittent failure of
> {{ITestS3AContractRootDir.testRmEmptyRootDirNonRecursive}} surfacing in
> HADOOP-12774 test run.
> This is a test which came in with HADOOP-12977, one test which deletes all
> children of the root dir, then verifies that they are gone. Although it
> tested happily during development, the sightings of two transient failures
> before it worked implied that it's either got some race condition with
> previous tests and/or maven build, or we are seeing listing inconsistency.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
