[GitHub] spark pull request #23081: [SPARK-26109][WebUI]Duration in the task summary ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23081 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23081: [SPARK-26109][WebUI]Duration in the task summary ...
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23081#discussion_r235075045 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -996,7 +996,7 @@ private[ui] object ApiHelper { HEADER_EXECUTOR -> TaskIndexNames.EXECUTOR, HEADER_HOST -> TaskIndexNames.HOST, HEADER_LAUNCH_TIME -> TaskIndexNames.LAUNCH_TIME, -HEADER_DURATION -> TaskIndexNames.DURATION, +HEADER_DURATION -> TaskIndexNames.EXEC_RUN_TIME, --- End diff -- Thanks. Done --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23081: [SPARK-26109][WebUI]Duration in the task summary ...
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23081#discussion_r235075470 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -996,7 +996,7 @@ private[ui] object ApiHelper { HEADER_EXECUTOR -> TaskIndexNames.EXECUTOR, HEADER_HOST -> TaskIndexNames.HOST, HEADER_LAUNCH_TIME -> TaskIndexNames.LAUNCH_TIME, -HEADER_DURATION -> TaskIndexNames.DURATION, +HEADER_DURATION -> TaskIndexNames.EXEC_RUN_TIME, --- End diff -- Done. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23081: [SPARK-26109][WebUI]Duration in the task summary ...
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/23081#discussion_r235071529 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -996,7 +996,7 @@ private[ui] object ApiHelper { HEADER_EXECUTOR -> TaskIndexNames.EXECUTOR, HEADER_HOST -> TaskIndexNames.HOST, HEADER_LAUNCH_TIME -> TaskIndexNames.LAUNCH_TIME, -HEADER_DURATION -> TaskIndexNames.DURATION, +HEADER_DURATION -> TaskIndexNames.EXEC_RUN_TIME, --- End diff -- Not big deal but there are two spaces after `SPARK-26109:` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23081: [SPARK-26109][WebUI]Duration in the task summary ...
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23081#discussion_r235069217 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -996,7 +996,7 @@ private[ui] object ApiHelper { HEADER_EXECUTOR -> TaskIndexNames.EXECUTOR, HEADER_HOST -> TaskIndexNames.HOST, HEADER_LAUNCH_TIME -> TaskIndexNames.LAUNCH_TIME, -HEADER_DURATION -> TaskIndexNames.DURATION, +HEADER_DURATION -> TaskIndexNames.EXEC_RUN_TIME, --- End diff -- Thanks. Done --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23081: [SPARK-26109][WebUI]Duration in the task summary ...
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/23081#discussion_r235065240 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -996,7 +996,7 @@ private[ui] object ApiHelper { HEADER_EXECUTOR -> TaskIndexNames.EXECUTOR, HEADER_HOST -> TaskIndexNames.HOST, HEADER_LAUNCH_TIME -> TaskIndexNames.LAUNCH_TIME, -HEADER_DURATION -> TaskIndexNames.DURATION, +HEADER_DURATION -> TaskIndexNames.EXEC_RUN_TIME, --- End diff -- Nit: add comment to explain why it should be executor run time here --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23081: [SPARK-26109][WebUI]Duration in the task summary ...
GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/23081 [SPARK-26109][WebUI]Duration in the task summary metrics table and the task table are different ## What changes were proposed in this pull request? Task summary displays the summary of the task table in the stage page. However, the duration metrics of task summary and task table are not matching. The reason is because, in the task summary we display executorRunTime as the duration and in task table, the actual duration. Except duration metrics, all other metrics are properly displaying in the task summary. In Spark2.2, we used to show executorRunTime as duration in the taskTable. That is why, in summary metrics also the exeuctorRunTime shows as the duration. In Spark2.3, it changed to the actual duration of task. So, summary metrics also should change according to that. ## How was this patch tested? Before patch: ![screenshot from 2018-11-19 04-32-06](https://user-images.githubusercontent.com/23054875/48679263-1e4fff80-ebb4-11e8-9ed5-16d892039e01.png) After patch: ![screenshot from 2018-11-19 04-37-39](https://user-images.githubusercontent.com/23054875/48679343-e39a9700-ebb4-11e8-8df9-9dc3a28d4bce.png) You can merge this pull request into a Git repository by running: $ git pull https://github.com/shahidki31/spark duratinSummary Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/23081.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #23081 commit 131164c2104a119468e782fb1d484f2d15274e33 Author: Shahid Date: 2018-11-18T22:38:21Z taskMetrics duration --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org