[
https://issues.apache.org/jira/browse/GEARPUMP-149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15303591#comment-15303591
]
Sean Zhong commented on GEARPUMP-149:
-------------------------------------
The task id is stable as long as the DAG is the same, no many how many times
you submitted it as an application. So may be we can develop to tool to
generate the task location config on behalf of the user.
> Binding tasks to workers
> ------------------------
>
> Key: GEARPUMP-149
> URL: https://issues.apache.org/jira/browse/GEARPUMP-149
> Project: Apache Gearpump
> Issue Type: Improvement
> Reporter: Karol Brejna
> Priority: Minor
>
> Let's assume that we have a cluster with some "specialized" nodes (for
> example a worker on IoT device that is responsible for controlling some
> actuators, dedicated workers to achieve data locality, some hardware specific
> - mem/cpu - tasks, etc.).
> There should be a way to "hint" Gearpump to prefer some nodes (workers) over
> others for certain tasks (executors).
> We already have "localities" feature. Developer can add a configuration (when
> submitting the application) to bind a task with a worker:
> {code}
> gearpump {
> streaming {
> localities {
> app1: { //This is the application name to be submitted
> workerId: [ //The worker id where to launch Task0_0 and Task0_1
> TaskId(0,0), TaskId(0,1)
> ]
> }
> }
> }
> }
> {code}
> The problem is: users have no idea what each Task's task ID is (especially if
> there are lots of nodes in a DAG).
> We could introduce more flexible "binding" mechanism, for example labeling
> tasks and workers or using some other "awareness" algorithm.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
