GitHub user shahidki31 opened a pull request:
https://github.com/apache/spark/pull/23081
[SPARK-26109][WebUI]Duration in the task summary metrics table and the task
table are different
## What changes were proposed in this pull request?
Task summary displays the summary of the task table in the stage page.
However, the duration metrics of task summary and task table are not matching.
The reason is because, in the task summary we display executorRunTime as the
duration and in task table, the actual duration.
Except duration metrics, all other metrics are properly displaying in the
task summary.
In Spark2.2, we used to show executorRunTime as duration in the taskTable.
That is why, in summary metrics also the exeuctorRunTime shows as the duration.
In Spark2.3, it changed to the actual duration of task. So, summary metrics
also should change according to that.
## How was this patch tested?
Before patch:
After patch:
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shahidki31/spark duratinSummary
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/23081.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #23081
----
commit 131164c2104a119468e782fb1d484f2d15274e33
Author: Shahid <shahidki31@...>
Date: 2018-11-18T22:38:21Z
taskMetrics duration
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
