Repository: spark Updated Branches: refs/heads/master 1b5ee2003 -> 42ad93b2c
[SPARK-19384][ML] forget unpersist input dataset in IsotonicRegression ## What changes were proposed in this pull request? unpersist the input dataset if `handlePersistence` = true ## How was this patch tested? existing tests Author: Zheng RuiFeng <[email protected]> Closes #16718 from zhengruifeng/isoReg_unpersisit. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/42ad93b2 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/42ad93b2 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/42ad93b2 Branch: refs/heads/master Commit: 42ad93b2c9047a68c14cbf681508157101f43c0e Parents: 1b5ee20 Author: Zheng RuiFeng <[email protected]> Authored: Sat Jan 28 10:18:47 2017 +0000 Committer: Sean Owen <[email protected]> Committed: Sat Jan 28 10:18:47 2017 +0000 ---------------------------------------------------------------------- .../scala/org/apache/spark/ml/regression/IsotonicRegression.scala | 2 ++ 1 file changed, 2 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/42ad93b2/mllib/src/main/scala/org/apache/spark/ml/regression/IsotonicRegression.scala ---------------------------------------------------------------------- diff --git a/mllib/src/main/scala/org/apache/spark/ml/regression/IsotonicRegression.scala b/mllib/src/main/scala/org/apache/spark/ml/regression/IsotonicRegression.scala index 90e77bc..a6c2943 100644 --- a/mllib/src/main/scala/org/apache/spark/ml/regression/IsotonicRegression.scala +++ b/mllib/src/main/scala/org/apache/spark/ml/regression/IsotonicRegression.scala @@ -175,6 +175,8 @@ class IsotonicRegression @Since("1.5.0") (@Since("1.5.0") override val uid: Stri val isotonicRegression = new MLlibIsotonicRegression().setIsotonic($(isotonic)) val oldModel = isotonicRegression.run(instances) + if (handlePersistence) instances.unpersist() + val model = copyValues(new IsotonicRegressionModel(uid, oldModel).setParent(this)) instr.logSuccess(model) model --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
