[
https://issues.apache.org/jira/browse/MAPREDUCE-205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J reopened MAPREDUCE-205:
-------------------------------
Assignee: (was: Owen O'Malley)
In hindsight, I think this was valid but it still has some stale information
and the discussion would be better moved to a new ticket so that it fits in
with the MR2 model we have in trunk/0.23 now, 'stead of the older protocols.
> Add ability to send "signals" to jobs and tasks
> -----------------------------------------------
>
> Key: MAPREDUCE-205
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-205
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Reporter: Andrzej Bialecki
>
> In some cases it would be useful to be able to "signal" a job and its tasks
> about some external condition, or to broadcast a specific message to all
> tasks in a job. Currently we can only send a single pseudo-signal, that is
> to kill a job.
> Example 1: some jobs may be gracefully terminated even if they didn't
> complete all their work, e.g. Fetcher in Nutch may be running for a very long
> time if it blocks on relatively few sites left over from the fetchlist. In
> such case it would be very useful to send it a message requesting that it
> discards the rest of its input and gracefully completes its map tasks.
> Example 2: available bandwidth for fetching may be different at different
> times of day, e.g. daytime vs. nighttime, or total external link usage by
> other applications. Fetcher jobs often run for several hours. It would be
> good to be able to send a "signal" to the Fetcher to throttle or un-throttle
> its bandwidth usage depending on external conditions.
> Job implementations could react to these messages either by implementing a
> method, or by registering a listener, whichever seems more natural.
> I'm not quite sure how to go about implementing it, I guess this would have
> to be a part of TaskUmbilicalProtocol but my knowledge here is a bit fuzzy
> ... ;) Comments are welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
