+1 for pluggable feature/ hadoop's counter arch
On 25 November 2011 01:05, Thomas Jungblut <[email protected]> wrote: > Great, I open an issue for this counter stuff. > > 2011/11/24 Edward J. Yoon <[email protected]> > >> >> I think it would be good as a 'pluggable' feature as this counting >> could>> impact performances. >> and >> >> > It is difficult to say whether we actually need this.* >> > *In case of SSSP it is useful, but as Tommaso says it can degrade >> >> I agree with you. If possible, let's provide as a optional feature. >> >> > What do you think of a similar architecture of Counters like in Hadoop? >> >> Good. >> >> Currently, our Job client reports only shows the total number of >> supersteps. I think that the Job reports needs to be strengthened. :-) >> >> On Thu, Nov 24, 2011 at 11:05 PM, Thomas Jungblut >> <[email protected]> wrote: >> > It is difficult to say whether we actually need this.* >> > *In case of SSSP it is useful, but as Tommaso says it can degrade >> > performance for every other BSP by increasing the cost of the sync. >> > >> > What do you think of a similar architecture of Counters like in Hadoop? >> > This could contain even more information, e.G. the number of bytes >> > transferred etc. >> > >> > 2011/11/24 Tommaso Teofili <[email protected]> >> > >> >> Hello Edward, >> >> >> >> I think it would be good as a 'pluggable' feature as this counting could >> >> impact performances. >> >> So it'd be useful when explicitly "debugging" while it should be, in my >> >> opinion, disabled by default. >> >> >> >> My 2 cents. >> >> Tommaso >> >> >> >> >> >> 2011/11/24 Edward J. Yoon <[email protected]> >> >> >> >> > Hi all, >> >> > >> >> > I'd like to add counting feature to get the total number of message >> >> > exchanges during one synchronization among BSP tasks. Before leave >> >> > barrier in sync() method, each task can report their counts. >> >> > >> >> > This information will be useful to user and, some applications (for >> >> > example, the sync cost of current SSSP example can be reduced to 1/3). >> >> > Should we add this feature? >> >> > >> >> > -- >> >> > Best Regards, Edward J. Yoon >> >> > @eddieyoon >> >> > >> >> >> > >> > >> > >> > -- >> > Thomas Jungblut >> > Berlin <[email protected]> >> > >> >> >> >> -- >> Best Regards, Edward J. Yoon >> @eddieyoon >> > > > > -- > Thomas Jungblut > Berlin <[email protected]> >
