[
https://issues.apache.org/jira/browse/MAPREDUCE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13107921#comment-13107921
]
Alejandro Abdelnur commented on MAPREDUCE-3028:
-----------------------------------------------
@Robert, with MR1 we are doing push. A lesson learned from that is that if the
receivers of the call are slow or not avail the push notifications pile up and
there is a long delay before they are delivered (at 60 secs HTTP timeout per
bad notification, this is a lot in a heavy loaded system). Using a RSS (like)
feed the amount of memory used in the JT is about the same (I would think even
less) and there is not contention & delay. Plus the fact that more than one
party can 'poll' for the status of a job as opposed to a single party 'push'.
Using something like size-bound concurrent list to store & serve the job status
would not create contention in the JT. Clients could poll often (30 to 60 secs)
which seems like a minimal delay.
> Support job end notification in .next /0.23
> -------------------------------------------
>
> Key: MAPREDUCE-3028
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3028
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Mohammad Kamrul Islam
> Fix For: 0.23.0
>
>
> Oozie primarily depends on the job end notification to determine when the
> job finishes. In the current version, job end notification is implemented in
> job tracker. Since job tracker will be removed in the upcoming hadoop release
> (.next), we wander where this support will move. I think this best effort
> notification could be implemented in the new Application Manager as one of
> the last step of job completion.
> Whatever implementation will it be, Oozie badly needs this feature to be
> continued in next releases as well.
>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
