GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/23038
[SPARK-25451][CORE][WEBUI]Aggregated metrics table doesn't show the right number of the total tasks ## What changes were proposed in this pull request? Total tasks in the aggregated table and the tasks table are not matching some times in the WEBUI. We need to update the executor summary of particular executor, when ever last task of that executor has reached. Currently it update based on last task, of all the executor. So, some particular executor task might miss, because for live application we update after some period. ## How was this patch tested? Tests to reproduce: ``` bin/spark-shell --master yarn --conf spark.executor.instances=3 sc.parallelize(1 to 10000, 10).map{ x => throw new RuntimeException("Bad executor")}.collect() ``` Before patch: ![screenshot from 2018-11-15 02-24-05](https://user-images.githubusercontent.com/23054875/48511776-b0d36480-e87d-11e8-89a8-ab97216e2c21.png) After patch: ![screenshot from 2018-11-15 02-32-38](https://user-images.githubusercontent.com/23054875/48512141-c39a6900-e87e-11e8-8535-903e1d11d13e.png) You can merge this pull request into a Git repository by running: $ git pull https://github.com/shahidki31/spark SPARK-25451 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/23038.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #23038 ---- commit ed9895827b69e5f47e4fbe93de479663c47a4a37 Author: Shahid <shahidki31@...> Date: 2018-11-13T22:24:04Z Stages page doesn't show the right number of the total tasks ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org