[
https://issues.apache.org/jira/browse/HADOOP-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12634820#action_12634820
]
Hemanth Yamijala commented on HADOOP-4053:
------------------------------------------
Owen, the jobUpdated API currently passes the JobInProgress argument. If this
is called at every state change, the listeners would need to keep track of what
actually changed. I think that would be a bit expensive, because it would mean
the schedulers need to keep a copy of the old state doubling the memory
requirements.
A better API would have been something like {{jobStateChanged(ChangeEvent e)}}
where the ChangeEvent captures the state transition in some way. This would
keep it generic and allow for additional state transitions to be added in the
future. If done this way, it would solve the problem.
Looking at the way it currently is implemented, it seems like the intent is to
capture each state change via a separate call, and somehow the job completion
event was missed (or thought unnecessary). In that case, possibly adding
jobCompleted is the right approach. This does have the problem that additional
state changes that may come in future (for e.g. paused ?) may need changes to
the interface again.
So, either way, I think we would need to change the current interface a bit.
And it seems like adding jobCompleted is less intrusive at this stage. Comments
?
> Schedulers need to know when a job has completed
> ------------------------------------------------
>
> Key: HADOOP-4053
> URL: https://issues.apache.org/jira/browse/HADOOP-4053
> Project: Hadoop Core
> Issue Type: Improvement
> Affects Versions: 0.19.0
> Reporter: Vivek Ratan
> Assignee: Amar Kamat
> Fix For: 0.19.0
>
> Attachments: HADOOP-4053-v1.patch, HADOOP-4053-v2.patch
>
>
> The JobInProgressListener interface is used by the framework to notify
> Schedulers of when jobs are added, removed, or updated. Right now, there is
> no way for the Scheduler to know that a job has completed. jobRemoved() is
> called when a job is retired, which can happen many hours after a job is
> actually completed. jobUpdated() is called when a job's priority is changed.
> We need to notify a listener when a job has completed (either successfully,
> or has failed or been killed).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
