How about we add a method + public String getPeerName(int index);
to BSPPeer class? It's hard to specify hostname for some special task e.g., master task, secondary task in a job configuration because we'll provide automated task scheduler. On Fri, Oct 14, 2011 at 8:48 PM, Thomas Jungblut <[email protected]> wrote: > Hey thanks Edward, > > the worse thing is that we are the user (in case of the examples). > So do we want to provide a function for that? > We could use the getAllPeerNames()[0] as a master task, but Zookeeper is not > consistent with the ordering of the peers, so there would be collisions. > > Or is there a master-election algorithm for BSP?:P > > 2011/10/14 Edward J. Yoon <[email protected]> > >> IMO, user have to elect master task in bsp() function, considering >> advanced job scheduler for concurrent jobs and multi-user system. >> >> On Thu, Oct 13, 2011 at 9:05 PM, Thomas Jungblut >> <[email protected]> wrote: >> > Hey all, >> > >> > I (theoretically) faced a problem with our examples that require a master >> > task. e.G. PiEstimator, SSSP/Pagerank and stuff. >> > Looking at the code, the cluster status which is used to "elect" a master >> > task returns plainly the groom names. >> > Since peers are now created per task and not globally for a single groom, >> > these names won't match anymore. >> > The same problem exists with YARN, since we don't have the information >> how >> > these tasks are named, and currently now ClusterStatus is supported. >> > >> > My first question, is this just a theoretical worry of me? >> > If not, my proposal would be to pre-launch a task in the cluster, get its >> > name and put it with a generic conf keyword (bsp.master.task) into >> > configuration. >> > Then launch the other tasks with the "new" configuration. >> > >> > -- >> > Thomas Jungblut >> > Berlin <[email protected]> >> > >> >> >> >> -- >> Best Regards, Edward J. Yoon >> @eddieyoon >> > > > > -- > Thomas Jungblut > Berlin <[email protected]> > -- Best Regards, Edward J. Yoon @eddieyoon
