[
https://issues.apache.org/jira/browse/MAPREDUCE-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438029#comment-16438029
]
Xuan Gong commented on MAPREDUCE-7042:
--------------------------------------
[~rohithsharma]
The basic workflow will be:
* When we got the JHS event, we push the event to JHS event queue.
* We have separate JHS event handler thread periodically pick and handle the
event from JHS event queue.
* When we start to handle the event, if ATS is enabled, we could push the
event immediately into ATS event queue.
* We have separate ATS event handler thread periodically pick and handle the
event from ATS event queue.
To separate JHS event and ATS event here, I think that I need two different
queues and handler threads. So, when we handle the event, we can simply push
the event into ATS event queue and continue to handle this as JHS event. So,
even there is anything wrong with ATS, it would not block the process of
handling JHS event.
> Killed mapreduce job data does not move to mapreduce.jobhistory.done-dir when
> ATS v2 is enabled
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-7042
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7042
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 3.0.0
> Reporter: Yesha Vora
> Assignee: Xuan Gong
> Priority: Major
> Attachments: MAPREDUCE-7042.1.patch
>
>
> Steps:
> 1) Start a mapreduce job
> {code}
> hadoop jar
> /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient-3.0.0.3.0.0.0-751-tests.jar
> sleep "-Dmapreduce.job.user.name=hrt_qa" -m 10 -r 1 -mt 1000 -rt
> 1000{code}
> 2) kill job
> 3) Validate job is present at mapreduce.jobhistory.done-dir
> mapreduce.jobhistory.done-dir dir does not have job_1516776025831_0048 entry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
