GitHub user MechCoder opened a pull request:
https://github.com/apache/spark/pull/14273
[SPARK-9140] [ML] Replace TimeTracker by MultiStopwatch
## What changes were proposed in this pull request?
Builds upon the work done by @hhbyyh in
https://github.com/apache/spark/pull/7871 . This replaces all occurrences of
TimeTracker with the more useful MultiStopWatch. More useful because it is
possible to bench the total time across the worker nodes as well, for instance
in the method `binsToBestSplit` using the `DistributedStopwatch`. It is also
very useful to measure the optimizations in terms of time done in
https://github.com/apache/spark/pull/13959 and should be merged before that
gets reviewed. It also removes the `TimeTracker` since it is not being used
elsewhere except the tree module.
## How was this patch tested?
It was run using `setLogLevel("INFO")` and the following timings are
printed out.
16/07/19 16:45:18 INFO RandomForest: {
binsToBestSplit: 26ms,
chooseSplits: 301ms,
findBestSplits: 307ms,
findSplitsBins: 553ms,
init: 1229ms,
total: 1572ms
}
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/MechCoder/spark timeTracker
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14273.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14273
----
commit fc055532ff3afac0df14e5ff8b63358f9410eae6
Author: Yuhao Yang <hhb...@gmail.com>
Date: 2015-08-02T16:25:30Z
Initial draft
commit c981ad554fa4706fbda40b42acbe4b275a2dbf47
Author: MechCoder <mks...@nyu.edu>
Date: 2016-07-19T21:50:12Z
Remove unused import
commit ea9caf497f392b9149572a2ff4fcefed9d66f9ab
Author: MechCoder <mks...@nyu.edu>
Date: 2016-07-19T22:32:19Z
Add MultiStopWatch to GBT's
commit 7cb2fa09232f8512b018e0673d9b2d4402f88c86
Author: MechCoder <mks...@nyu.edu>
Date: 2016-07-19T22:33:37Z
Remove TimeTracker
commit 3dd9b3135722aa937b04052501876dc2b3ebb06f
Author: MechCoder <mks...@nyu.edu>
Date: 2016-07-19T23:21:50Z
Pass MultiStopWatch instead of LocalStopWatch
commit e5b077de8a901bae666ff25d2e1800caf622681b
Author: MechCoder <mks...@nyu.edu>
Date: 2016-07-19T23:48:51Z
add distributed timer to multitimer
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
