Huafeng, Manu - thank you for quick response. As for Task id, I agree it's not ideal :) Let me read the document to get better understanding of the matter.
Karol śr., 11.05.2016 o 10:19 użytkownik Manu Zhang <[email protected]> napisał: > Hi Karol, > > Weihua has previously written up this nice doc IoT features map with > Gearpump > < > https://docs.google.com/document/d/1kagryYsBmtwVIE5E8afgH9K_O2a0_VdtlAY9auVAQQM/edit?usp=sharing > >. > You may find more details about how Gearpump can work in the environment of > cluster and edge devices and the gaps to fill. > > On Wed, May 11, 2016 at 4:11 PM Vincent Wang <[email protected]> wrote: > > > You can add a configuration when submitting the application, the format > is: > > > > gearpump { > > streaming { > > localities { > > app1: { //This is the application name to be submitted > > workerId: [ //The worker id where to launch Task0_0 and > Task0_1 > > TaskId(0,0), TaskId(0,1) > > ] > > } > > } > > } > > } > > > > But I think the current way is not flexible when there are lots of nodes > in > > a DAG because user have no idea what each Task's task ID is. > > > > Thanks, > > Huafeng > > > > 2016-05-11 15:49 GMT+08:00 Karol Brejna <[email protected]>: > > > > > What are the criteria that decide which Executor/Tasks goes to which > > > Workers right now? > > > > > > I am considering the following scenario. > > > > > > Let's assume that we have a cluster with some "specialized" nodes (for > > > example a worker on IoT device that is responsible for controlling some > > > actuators, dedicated workers to achieve data locality, some hardware > > > specific - mem/cpu - tasks, etc.). > > > > > > Are we able to "hint" Gearpump for certain tasks to prefer some nodes > > over > > > others? (For example by labeling tasks and nodes or using some other > > > "awareness" mechanism?) > > > > > > Regards, > > > Karol > > > > > >
