[
https://issues.apache.org/jira/browse/HADOOP-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved HADOOP-15240.
-------------------------------------
Resolution: Cannot Reproduce
> ITestS3AEmptyDirectory.testParallelJobsToAdjacentPaths failure,
> FileAlreadyExistsException
> ------------------------------------------------------------------------------------------
>
> Key: HADOOP-15240
> URL: https://issues.apache.org/jira/browse/HADOOP-15240
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 3.1.0, 3.2.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
>
> Transient {{ITestS3AEmptyDirectory.testParallelJobsToAdjacentPaths}} failure;
> {{FileAlreadyExistsException}} during the setup phase of the test -writing
> out the work to
> {{org.apache.hadoop.fs.FileAlreadyExistsException: File already exists:
> file:/Users/stevel/Projects/hadoop-trunk/hadoop-tools/hadoop-aws/target/build/test:fork-0002/job_200707120153_0002/_temporary/1/_temporary/attempt_200707120153_0002_m_000000_0/part-m-00000}}
> There's something wrong here as that path is a FileOutputCommitter workingDir
> path, not a partitioned committer.
> Hypotheses: some filesystem caching in the test run is stopping the right
> committer from being picked up.
> It's possible that this has been happening for a while, but only some cleanup
> failure from a previous run has caught it.
> Proposed
> * add checks in the AbstractIT test to verify the right committer is picked
> up in the task context by probing for the working dir: file:// for staging,
> */__magic/* for magic.
> * and verify that success file is there.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
