[
https://issues.apache.org/jira/browse/YARN-5401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15384330#comment-15384330
]
Jason Lowe commented on YARN-5401:
----------------------------------
This is effectively a duplicate of YARN-2261. MapReduce history requires the
MapReduce ApplicationMaster to generate the history when it completes. hadoop
job -kill or mapred job -kill accomplishes the kill by having the client
connect to the MapReduce ApplicationMaster for the job and asks it to kill the
job. Since this goes through the ApplicationMaster it allows the history to be
generated properly.
When the kill is done via YARN then the ApplicationMaster is not involved. The
ResourceManager kills the AM without the AM's knowledge. This is similar to
kill vs. kill -9 (i.e.: SIGTERM vs SIGKILL) in POSIX. The former allows the
application to perform cleanup tasks on the way down, while the latter
mercilessly kills the process without any chance for cleanup.
Since YARN does not allow the application to specify a cleanup task to be
performed when the app dies the MapReduce framework doesn't get a chance to
finish generating the history for the job.
> yarn application kill does not let mapreduce jobs show up in jobhistory
> -----------------------------------------------------------------------
>
> Key: YARN-5401
> URL: https://issues.apache.org/jira/browse/YARN-5401
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Environment: centos 6.6
> apache hadoop 2.6.4
> Reporter: Nikhil Mulley
>
> Hi,
> Its been found in our cluster running apache hadoop 2.6.4, that while the
> mapreduce jobs that are killed with 'hadoop job -kill' command do end up have
> the job and its counters to jobhistory server but when 'yarn application
> -kill' is used on mapreduce application, job does not show up in jobhistory
> server interface.
> Is this intentional? If so, any particular reasons?
> It would be better to have mapreduce application history reported on
> jobhistory irrespective of whether kill is performed using yarn application
> cli or hadoop job cli.
> thanks,
> Nikhil
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
