[
https://issues.apache.org/jira/browse/HADOOP-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638729#action_12638729
]
Amar Kamat commented on HADOOP-4053:
------------------------------------
I had an offline discussion with Hemanth and Devaraj and here is what we all
agreed upon :
bq. Do we need JobStatusChangeEvent(JobInProgress jip, EventType eventType,
JobStatus status) ?
No. Job's state is completely manipulated and controlled by an external entity
(JobTracker, JobInProgressListener etc) and hence we should always have an
_old_ and a _new_ state. Job's state changes via well known apis and hence we
should capture the state changes whenever these apis are invoked. Following is
the list of knows events w.r.t to a job
||event-type||api||
|Job added|JIP constructor|
|Job inited|JIP.initTasks()|
|Job priority change|JobTracker.setJobPriority()|
|Running Job killed|JobTracker's heartbeat (cleanup tasks)|
|Queued job killed|JobTracker.killJob() --> JIP.terminateJob()|
|Job completed|JobTracker's heartbeat|
|Job removed|JobTracker.finalizeJob()/JobTracker's job expiry thread|
As of now the following changes are tracked
- job added
- job inited
- job priority change
We just need to track the job-state change in:
- JobTracker.updateTaskStatuses() (job-completion, running-job-kill
[external-kill-leading-to-cleanup-tasks, too-many-failures])
- JobTracker.killJob() (external-queued-job-kill)
bq. Seems OK to have JonInProgress.initTasks() do the event notification
We cant just change the method signature as its a public api. Hence I
introduced {{JobInProgress.initTasksAndReportChange()}}. Looking at Hemanth's
comment#2
[here|https://issues.apache.org/jira/browse/HADOOP-4053?focusedCommentId=12638478#action_12638478],
I feel its ok to make the init-event notification inline and whereever needed.
We can factor it out later, if needed.
> Schedulers need to know when a job has completed
> ------------------------------------------------
>
> Key: HADOOP-4053
> URL: https://issues.apache.org/jira/browse/HADOOP-4053
> Project: Hadoop Core
> Issue Type: Improvement
> Affects Versions: 0.19.0
> Reporter: Vivek Ratan
> Assignee: Amar Kamat
> Priority: Blocker
> Attachments: HADOOP-4053-v1.patch, HADOOP-4053-v2.patch,
> HADOOP-4053-v3.1.patch, HADOOP-4053-v3.2.patch, HADOOP-4053-v4.1.patch,
> HADOOP-4053-v4.patch
>
>
> The JobInProgressListener interface is used by the framework to notify
> Schedulers of when jobs are added, removed, or updated. Right now, there is
> no way for the Scheduler to know that a job has completed. jobRemoved() is
> called when a job is retired, which can happen many hours after a job is
> actually completed. jobUpdated() is called when a job's priority is changed.
> We need to notify a listener when a job has completed (either successfully,
> or has failed or been killed).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
