Hey thanks Edward, the worse thing is that we are the user (in case of the examples). So do we want to provide a function for that? We could use the getAllPeerNames()[0] as a master task, but Zookeeper is not consistent with the ordering of the peers, so there would be collisions.
Or is there a master-election algorithm for BSP?:P 2011/10/14 Edward J. Yoon <[email protected]> > IMO, user have to elect master task in bsp() function, considering > advanced job scheduler for concurrent jobs and multi-user system. > > On Thu, Oct 13, 2011 at 9:05 PM, Thomas Jungblut > <[email protected]> wrote: > > Hey all, > > > > I (theoretically) faced a problem with our examples that require a master > > task. e.G. PiEstimator, SSSP/Pagerank and stuff. > > Looking at the code, the cluster status which is used to "elect" a master > > task returns plainly the groom names. > > Since peers are now created per task and not globally for a single groom, > > these names won't match anymore. > > The same problem exists with YARN, since we don't have the information > how > > these tasks are named, and currently now ClusterStatus is supported. > > > > My first question, is this just a theoretical worry of me? > > If not, my proposal would be to pre-launch a task in the cluster, get its > > name and put it with a generic conf keyword (bsp.master.task) into > > configuration. > > Then launch the other tasks with the "new" configuration. > > > > -- > > Thomas Jungblut > > Berlin <[email protected]> > > > > > > -- > Best Regards, Edward J. Yoon > @eddieyoon > -- Thomas Jungblut Berlin <[email protected]>
