[
https://issues.apache.org/jira/browse/MAPREDUCE-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14350842#comment-14350842
]
Gera Shegalov commented on MAPREDUCE-4815:
------------------------------------------
Hi [~jlowe], thanks for feedback! I don't think the upgrade problem is dubious.
Scenario is this:
# Cluster with old code, no version is set in site/job.xml. Jobs are running.
# Shutdown the cluster: all running app attempts failing
# Part of upgrade the cluster, yay we can use version=2, make it part of
cluster-wide mapred-site.xml
I understand it may not the best way to do it, but it's possible.
My biggest concern is that if people push the config first, and then start a
rolling upgrade, it may lead to an incomplete output because in commitJob we
rightfully decided to cut the legacy inefficiency.
# rolling upgrade, version=2 is pushed to cluster mapred-site
# AM is killed because its node is upgraded.
# AM restarts on some upgraded node and version=2
# Some outstanding task X is scheduled on old node, commits in a v1 fashion,
despite version=2
# Job completes, AM executes v2 commitJob and X output will be missing
> FileOutputCommitter.commitJob can be very slow for jobs with many output files
> ------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4815
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4815
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.3, 2.0.1-alpha, 2.4.1
> Reporter: Jason Lowe
> Assignee: Siqi Li
> Attachments: MAPREDUCE-4815.v10.patch, MAPREDUCE-4815.v11.patch,
> MAPREDUCE-4815.v12.patch, MAPREDUCE-4815.v13.patch, MAPREDUCE-4815.v14.patch,
> MAPREDUCE-4815.v15.patch, MAPREDUCE-4815.v16.patch, MAPREDUCE-4815.v17.patch,
> MAPREDUCE-4815.v3.patch, MAPREDUCE-4815.v4.patch, MAPREDUCE-4815.v5.patch,
> MAPREDUCE-4815.v6.patch, MAPREDUCE-4815.v7.patch, MAPREDUCE-4815.v8.patch,
> MAPREDUCE-4815.v9.patch
>
>
> If a job generates many files to commit then the commitJob method call at the
> end of the job can take minutes. This is a performance regression from 1.x,
> as 1.x had the tasks commit directly to the final output directory as they
> were completing and commitJob had very little to do. The commit work was
> processed in parallel and overlapped the processing of outstanding tasks. In
> 0.23/2.x, the commit is single-threaded and waits until all tasks have
> completed before commencing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
