Perhaps we can consider a new feature where a task label can be used instead of
the task id.
On Wednesday, May 11, 2016 1:11 AM, Vincent Wang <[email protected]>
wrote:
You can add a configuration when submitting the application, the format is:
gearpump {
streaming {
localities {
app1: { //This is the application name to be submitted
workerId: [ //The worker id where to launch Task0_0 and Task0_1
TaskId(0,0), TaskId(0,1)
]
}
}
}
}
But I think the current way is not flexible when there are lots of nodes in
a DAG because user have no idea what each Task's task ID is.
Thanks,
Huafeng
2016-05-11 15:49 GMT+08:00 Karol Brejna <[email protected]>:
> What are the criteria that decide which Executor/Tasks goes to which
> Workers right now?
>
> I am considering the following scenario.
>
> Let's assume that we have a cluster with some "specialized" nodes (for
> example a worker on IoT device that is responsible for controlling some
> actuators, dedicated workers to achieve data locality, some hardware
> specific - mem/cpu - tasks, etc.).
>
> Are we able to "hint" Gearpump for certain tasks to prefer some nodes over
> others? (For example by labeling tasks and nodes or using some other
> "awareness" mechanism?)
>
> Regards,
> Karol
>
