Hi Karol, Weihua has previously written up this nice doc IoT features map with Gearpump <https://docs.google.com/document/d/1kagryYsBmtwVIE5E8afgH9K_O2a0_VdtlAY9auVAQQM/edit?usp=sharing>. You may find more details about how Gearpump can work in the environment of cluster and edge devices and the gaps to fill.
On Wed, May 11, 2016 at 4:11 PM Vincent Wang <[email protected]> wrote: > You can add a configuration when submitting the application, the format is: > > gearpump { > streaming { > localities { > app1: { //This is the application name to be submitted > workerId: [ //The worker id where to launch Task0_0 and Task0_1 > TaskId(0,0), TaskId(0,1) > ] > } > } > } > } > > But I think the current way is not flexible when there are lots of nodes in > a DAG because user have no idea what each Task's task ID is. > > Thanks, > Huafeng > > 2016-05-11 15:49 GMT+08:00 Karol Brejna <[email protected]>: > > > What are the criteria that decide which Executor/Tasks goes to which > > Workers right now? > > > > I am considering the following scenario. > > > > Let's assume that we have a cluster with some "specialized" nodes (for > > example a worker on IoT device that is responsible for controlling some > > actuators, dedicated workers to achieve data locality, some hardware > > specific - mem/cpu - tasks, etc.). > > > > Are we able to "hint" Gearpump for certain tasks to prefer some nodes > over > > others? (For example by labeling tasks and nodes or using some other > > "awareness" mechanism?) > > > > Regards, > > Karol > > >
