spark git commit: [SPARK-20768][PYSPARK][ML] Expose numPartitions (expert) param of PySpark FPGrowth.
Repository: spark Updated Branches: refs/heads/branch-2.2 9cbf39f1c -> e01f1f222 [SPARK-20768][PYSPARK][ML] Expose numPartitions (expert) param of PySpark FPGrowth. ## What changes were proposed in this pull request? Expose numPartitions (expert) param of PySpark FPGrowth. ## How was this patch tested? + [x] Pass all unit tests. Author: Yan Facai (é¢åæ) Closes #18058 from facaiy/ENH/pyspark_fpg_add_num_partition. (cherry picked from commit 139da116f130ed21481d3e9bdee5df4b8d7760ac) Signed-off-by: Yanbo Liang Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e01f1f22 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/e01f1f22 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/e01f1f22 Branch: refs/heads/branch-2.2 Commit: e01f1f222bcb7c469b1e1595e9338ed478d99894 Parents: 9cbf39f Author: Yan Facai (é¢åæ) Authored: Thu May 25 21:40:39 2017 +0800 Committer: Yanbo Liang Committed: Thu May 25 21:40:52 2017 +0800 -- python/pyspark/ml/fpm.py | 30 +- 1 file changed, 29 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/spark/blob/e01f1f22/python/pyspark/ml/fpm.py -- diff --git a/python/pyspark/ml/fpm.py b/python/pyspark/ml/fpm.py index 6ff7d2c..dd7dda5 100644 --- a/python/pyspark/ml/fpm.py +++ b/python/pyspark/ml/fpm.py @@ -49,6 +49,32 @@ class HasMinSupport(Params): return self.getOrDefault(self.minSupport) +class HasNumPartitions(Params): +""" +Mixin for param numPartitions: Number of partitions (at least 1) used by parallel FP-growth. +""" + +numPartitions = Param( +Params._dummy(), +"numPartitions", +"Number of partitions (at least 1) used by parallel FP-growth. " + +"By default the param is not set, " + +"and partition number of the input dataset is used.", +typeConverter=TypeConverters.toInt) + +def setNumPartitions(self, value): +""" +Sets the value of :py:attr:`numPartitions`. +""" +return self._set(numPartitions=value) + +def getNumPartitions(self): +""" +Gets the value of :py:attr:`numPartitions` or its default value. +""" +return self.getOrDefault(self.numPartitions) + + class HasMinConfidence(Params): """ Mixin for param minConfidence. @@ -127,7 +153,9 @@ class FPGrowthModel(JavaModel, JavaMLWritable, JavaMLReadable): class FPGrowth(JavaEstimator, HasItemsCol, HasPredictionCol, - HasMinSupport, HasMinConfidence, JavaMLWritable, JavaMLReadable): + HasMinSupport, HasNumPartitions, HasMinConfidence, + JavaMLWritable, JavaMLReadable): + """ .. note:: Experimental - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
spark git commit: [SPARK-20768][PYSPARK][ML] Expose numPartitions (expert) param of PySpark FPGrowth.
Repository: spark Updated Branches: refs/heads/master 913a6bfe4 -> 139da116f [SPARK-20768][PYSPARK][ML] Expose numPartitions (expert) param of PySpark FPGrowth. ## What changes were proposed in this pull request? Expose numPartitions (expert) param of PySpark FPGrowth. ## How was this patch tested? + [x] Pass all unit tests. Author: Yan Facai (é¢åæ) Closes #18058 from facaiy/ENH/pyspark_fpg_add_num_partition. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/139da116 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/139da116 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/139da116 Branch: refs/heads/master Commit: 139da116f130ed21481d3e9bdee5df4b8d7760ac Parents: 913a6bf Author: Yan Facai (é¢åæ) Authored: Thu May 25 21:40:39 2017 +0800 Committer: Yanbo Liang Committed: Thu May 25 21:40:39 2017 +0800 -- python/pyspark/ml/fpm.py | 30 +- 1 file changed, 29 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/spark/blob/139da116/python/pyspark/ml/fpm.py -- diff --git a/python/pyspark/ml/fpm.py b/python/pyspark/ml/fpm.py index 6ff7d2c..dd7dda5 100644 --- a/python/pyspark/ml/fpm.py +++ b/python/pyspark/ml/fpm.py @@ -49,6 +49,32 @@ class HasMinSupport(Params): return self.getOrDefault(self.minSupport) +class HasNumPartitions(Params): +""" +Mixin for param numPartitions: Number of partitions (at least 1) used by parallel FP-growth. +""" + +numPartitions = Param( +Params._dummy(), +"numPartitions", +"Number of partitions (at least 1) used by parallel FP-growth. " + +"By default the param is not set, " + +"and partition number of the input dataset is used.", +typeConverter=TypeConverters.toInt) + +def setNumPartitions(self, value): +""" +Sets the value of :py:attr:`numPartitions`. +""" +return self._set(numPartitions=value) + +def getNumPartitions(self): +""" +Gets the value of :py:attr:`numPartitions` or its default value. +""" +return self.getOrDefault(self.numPartitions) + + class HasMinConfidence(Params): """ Mixin for param minConfidence. @@ -127,7 +153,9 @@ class FPGrowthModel(JavaModel, JavaMLWritable, JavaMLReadable): class FPGrowth(JavaEstimator, HasItemsCol, HasPredictionCol, - HasMinSupport, HasMinConfidence, JavaMLWritable, JavaMLReadable): + HasMinSupport, HasNumPartitions, HasMinConfidence, + JavaMLWritable, JavaMLReadable): + """ .. note:: Experimental - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
