IMO, user have to elect master task in bsp() function, considering advanced job scheduler for concurrent jobs and multi-user system.
On Thu, Oct 13, 2011 at 9:05 PM, Thomas Jungblut <[email protected]> wrote: > Hey all, > > I (theoretically) faced a problem with our examples that require a master > task. e.G. PiEstimator, SSSP/Pagerank and stuff. > Looking at the code, the cluster status which is used to "elect" a master > task returns plainly the groom names. > Since peers are now created per task and not globally for a single groom, > these names won't match anymore. > The same problem exists with YARN, since we don't have the information how > these tasks are named, and currently now ClusterStatus is supported. > > My first question, is this just a theoretical worry of me? > If not, my proposal would be to pre-launch a task in the cluster, get its > name and put it with a generic conf keyword (bsp.master.task) into > configuration. > Then launch the other tasks with the "new" configuration. > > -- > Thomas Jungblut > Berlin <[email protected]> > -- Best Regards, Edward J. Yoon @eddieyoon
