Accumulators on the stage info page show the rolling life time value of accumulators as well as per task which is handy. I think it would be useful to add another field to the “Accumulators” table that also shows the total for the stage you are looking at (basically just a merge of the accumulators for tasks in that stage). This would be useful for any job that is iterative (eg- basically every spark streaming job).
Does this idea make sense? Separate but related question- From the operational side I think it could be very useful to have an accumulators summary page. For example we have a spark streaming job with many different stages. It is difficult to navigate into each stage to pick out a trend. An accumulators page that allowed one to filter by stage description and/or accumulator name would be very useful. Thoughts? Thanks, Sean --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
