GitHub user kayousterhout opened a pull request:
https://github.com/apache/spark/pull/6484
[SPARK-7932] Fix misleading scheduler delay visualization
The existing code rounds down to the nearest percent when computing the
proportion
of a task's time that was spent on each phase of execution, and then
computes
the scheduler delay proportion as 100 - sum(all other proportions). As a
result,
a few extra percent can end up in the scheduler delay. This commit
eliminates
the rounding so that the time visualizations correspond properly to the
real times.
@sarutak If you could take a look at this, that would be great! Not sure if
there's a good
reason to round here that I missed.
cc @shivaram
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kayousterhout/spark-1 SPARK-7932
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/6484.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6484
----
commit 1723cc4270f094935684e69ba5fdbc6363b8a388
Author: Kay Ousterhout <[email protected]>
Date: 2015-05-29T01:51:24Z
[SPARK-7932] Fix misleading scheduler delay visualization
The existing code rounds down to the nearest percent when computing the
proportion
of a task's time that was spent on each phase of execution, and then
computes
the scheduler delay proportion as 100 - sum(all other proportions). As a
result,
a few extra percent can end up in the scheduler delay. This commit
eliminates
the rounding so that the time visualizations correspond properly to the
real times.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
