[
https://issues.apache.org/jira/browse/HADOOP-16207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16934667#comment-16934667
]
Siddharth Seth edited comment on HADOOP-16207 at 9/20/19 7:13 PM:
------------------------------------------------------------------
-Attached a simple patch which fixes just the test failures. Doesn't do
anything with parallelism, changing dir names to be different across tests etc.
Can submit this in a separate jira, if this one is being used for parallelizing
the tests.-
Switched to https://issues.apache.org/jira/browse/HADOOP-16591
was (Author: sseth):
Attached a simple patch which fixes just the test failures. Doesn't do anything
with parallelism, changing dir names to be different across tests etc. Can
submit this in a separate jira, if this one is being used for parallelizing the
tests.
> Fix ITestDirectoryCommitMRJob.testMRJob
> ---------------------------------------
>
> Key: HADOOP-16207
> URL: https://issues.apache.org/jira/browse/HADOOP-16207
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Critical
>
> Reported failure of {{ITestDirectoryCommitMRJob}} in validation runs of
> HADOOP-16186; assertIsDirectory with s3guard enabled and a parallel test run:
> Path "is recorded as deleted by S3Guard"
> {code}
> waitForConsistency();
> assertIsDirectory(outputPath) /* here */
> {code}
> The file is there but there's a tombstone. Possibilities
> * some race condition with another test
> * tombstones aren't timing out
> * committers aren't creating that base dir in a way which cleans up S3Guard's
> tombstones.
> Remember: we do have to delete that dest dir before the committer runs unless
> overwrite==true, so at the start of the run there will be a tombstone. It
> should be overwritten by a success.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
