[
https://issues.apache.org/jira/browse/MAPREDUCE-7378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jingpan xiong updated MAPREDUCE-7378:
-------------------------------------
Description:
When we use FileOutputCommitter as the base class of Job Committer for other
components, we may meet an concurrently writing problem.
Like HadoopMapReduceCommitProtocol in Spark, when there have multiple
application to write data in same path, they will commit job and task in the
"_temporary" dir. Once a Job finished ,it will delete the "_temporary" dir,
make the other jobs failed.
was:
When we use FileOutputCommitter as the base class of Job Committer for other
components, we may meet an concurrently writing problem.
Like HadoopMapReduceCommitProtocol in Spark, when there have multiple
application to write data in same path, they will commit job and task in the
"_temporary" dir. Once a Job finished ,it will delete the "_temporary" dir,
make the other jobs failed.
> An error occurred while concurrently writing to a path
> ------------------------------------------------------
>
> Key: MAPREDUCE-7378
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7378
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: jingpan xiong
> Priority: Major
>
> When we use FileOutputCommitter as the base class of Job Committer for other
> components, we may meet an concurrently writing problem.
> Like HadoopMapReduceCommitProtocol in Spark, when there have multiple
> application to write data in same path, they will commit job and task in the
> "_temporary" dir. Once a Job finished ,it will delete the "_temporary" dir,
> make the other jobs failed.
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
