GitHub user bien opened a pull request:
https://github.com/apache/spark/pull/5351
SPARK-6698: where RandomForest input specifies a storage level, honor it
internally
Currently RandomForest re-persists input data unconditionally with
StorageLevel.MEMORY_AND_DISK. This makes it difficult to tune GC with
different StorageLevels. Instead, if the input has already been persisted at a
different level, use that same level.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bien/spark master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5351.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5351
----
commit 93381814ce2fd76242a6a178af636406d6e12677
Author: Michael Bieniosek <[email protected]>
Date: 2015-04-03T16:49:29Z
SPARK-6698: if input specifies a storage level, honor it internally in
derived RDDs
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
