I wouldn't use accumulators for things which could get large, they can become kind of a bottle neck. Do you have a lot of string messages you want to bring back or only a few?
On Mon, Feb 1, 2016 at 3:24 PM, Utkarsh Sengar <utkarsh2...@gmail.com> wrote: > I am trying to debug code executed in executors by logging. Even when I > add log4j's LOG.info(..) inside .map() I don't see it in mesos task logs in > the corresponding slaves. > Its anyway inefficient to keep checking multiple slaves for logs. > > One way to deal with this is to push logs to a central location. > > Another way (for debugging purposes) is to use accumulators . Is it > advisable to use accumulators to push string from executors to driver? > It will simplify things when I am debugging datasets, bugs which is hard > to reproduce locally etc. > > Suggestions/comments? > > -Utkarsh > -- Cell : 425-233-8271 Twitter: https://twitter.com/holdenkarau
