Mustafa Iman created TEZ-4192:
---------------------------------
Summary: Attach task specific info in scheduler and retrieve in
communicator
Key: TEZ-4192
URL: https://issues.apache.org/jira/browse/TEZ-4192
Project: Apache Tez
Issue Type: Improvement
Affects Versions: 0.9.2
Reporter: Mustafa Iman
Assignee: Mustafa Iman
There is no way to pass information from scheduler to communicator. Sometimes
we want to let communicator know about extra properties of the task other than
its spec. One example is if a task is preemptable.
When scheduler receives a task, its taskspec is already constructed. It is a
nice immutable object at this point. We do not want to modify and put more info
in it at that stage. So we pass a new piece of information to
TaskSchedulerContext#taskAllocated along with the allocation. This info is
passed to the communicator untouched. It is up to the communicator
implementation to make use of it. Communicator implementation needs to override
a new registerRunningTaskAttempt that accepts this additional object along with
all the other arguments in the original registerRunningTaskAttempt method.
The new registerRunningTaskAttempt method has a default implementation that
falls back to the old method. So if the user does not override the new method,
the same old behavior is preserved. Similarly on scheduler side, the user needs
to call new taskAllocated method to pass the additional info object. If they
use the old taskAllocated method, old behavior is preserved.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
