[jira] [Updated] (SPARK-34429) KMeansSummary class is omitted from PySpark documentation

2021-02-12 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Bauer updated SPARK-34429: --- Summary: KMeansSummary class is omitted from PySpark documentation (was: KMeansSummary class is

[jira] [Created] (SPARK-34429) KMeansSummary class is omitted from PySPark documentation

2021-02-12 Thread John Bauer (Jira)
John Bauer created SPARK-34429: -- Summary: KMeansSummary class is omitted from PySPark documentation Key: SPARK-34429 URL: https://issues.apache.org/jira/browse/SPARK-34429 Project: Spark Issue

[jira] [Commented] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-11-14 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974527#comment-16974527 ] John Bauer commented on SPARK-29691: [[SPARK-29691] ensure Param objects are valid in fit,

[jira] [Commented] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-11-05 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967773#comment-16967773 ] John Bauer commented on SPARK-29691: Yes, I can do that.  An error message suggesting a call to

[jira] [Comment Edited] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-11-05 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967741#comment-16967741 ] John Bauer edited comment on SPARK-29691 at 11/5/19 6:13 PM: - I wonder if it

[jira] [Comment Edited] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-11-05 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967741#comment-16967741 ] John Bauer edited comment on SPARK-29691 at 11/5/19 6:11 PM: - I wonder if it

[jira] [Commented] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-11-05 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967741#comment-16967741 ] John Bauer commented on SPARK-29691: I wonder if it would make sense to do this: {code:java}

[jira] [Commented] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-11-04 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966941#comment-16966941 ] John Bauer commented on SPARK-29691: OK that works. I worked with fit doing a grid search some time

[jira] [Comment Edited] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-11-04 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966768#comment-16966768 ] John Bauer edited comment on SPARK-29691 at 11/4/19 5:20 PM: - I was using

[jira] [Comment Edited] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-11-04 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966768#comment-16966768 ] John Bauer edited comment on SPARK-29691 at 11/4/19 4:59 PM: - I was using

[jira] [Comment Edited] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-11-04 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966768#comment-16966768 ] John Bauer edited comment on SPARK-29691 at 11/4/19 4:57 PM: - I was using

[jira] [Comment Edited] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-11-04 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966768#comment-16966768 ] John Bauer edited comment on SPARK-29691 at 11/4/19 4:57 PM: - I was using

[jira] [Comment Edited] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-11-04 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966768#comment-16966768 ] John Bauer edited comment on SPARK-29691 at 11/4/19 4:56 PM: - I was using

[jira] [Comment Edited] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-11-04 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966768#comment-16966768 ] John Bauer edited comment on SPARK-29691 at 11/4/19 4:55 PM: - I was using

[jira] [Commented] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-11-04 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966768#comment-16966768 ] John Bauer commented on SPARK-29691: I will update the example shortly - I was using this in the

[jira] [Commented] (SPARK-12806) Support SQL expressions extracting values from VectorUDT

2019-10-31 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-12806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964481#comment-16964481 ] John Bauer commented on SPARK-12806: Also, when using PyArrow to convert a Spark DataFrame for use

[jira] [Commented] (SPARK-12806) Support SQL expressions extracting values from VectorUDT

2019-10-31 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-12806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964475#comment-16964475 ] John Bauer commented on SPARK-12806: This is still a problem. For example, classification models

[jira] [Updated] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-10-31 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Bauer updated SPARK-29691: --- Description: Estimator `fit` method is supposed to copy a dictionary of params, overwriting the

[jira] [Updated] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-10-31 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Bauer updated SPARK-29691: --- Description: Estimator `fit` method (implemented in Params) is supposed to copy a dictionary of

[jira] [Updated] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-10-31 Thread John Bauer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Bauer updated SPARK-29691: --- Description: Estimator `fit` method (implemented in Params) is supposed to copy a dictionary of

[jira] [Created] (SPARK-29691) Estimator fit method fails to copy params (in PySpark)

2019-10-31 Thread John Bauer (Jira)
John Bauer created SPARK-29691: -- Summary: Estimator fit method fails to copy params (in PySpark) Key: SPARK-29691 URL: https://issues.apache.org/jira/browse/SPARK-29691 Project: Spark Issue

[jira] [Comment Edited] (SPARK-17025) Cannot persist PySpark ML Pipeline model that includes custom Transformer

2019-06-04 Thread John Bauer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855948#comment-16855948 ] John Bauer edited comment on SPARK-17025 at 6/4/19 11:12 PM: - [~Hadar]

[jira] [Comment Edited] (SPARK-17025) Cannot persist PySpark ML Pipeline model that includes custom Transformer

2019-06-04 Thread John Bauer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855948#comment-16855948 ] John Bauer edited comment on SPARK-17025 at 6/4/19 11:10 PM: - [~Hadar]

[jira] [Commented] (SPARK-17025) Cannot persist PySpark ML Pipeline model that includes custom Transformer

2019-06-04 Thread John Bauer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855948#comment-16855948 ] John Bauer commented on SPARK-17025: [~Hadar] [~yug95] I wrote a minimal example of a PySpark 

[jira] [Comment Edited] (SPARK-21542) Helper functions for custom Python Persistence

2018-11-09 Thread John Bauer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681895#comment-16681895 ] John Bauer edited comment on SPARK-21542 at 11/9/18 8:07 PM: - Compared to

[jira] [Comment Edited] (SPARK-21542) Helper functions for custom Python Persistence

2018-11-09 Thread John Bauer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681895#comment-16681895 ] John Bauer edited comment on SPARK-21542 at 11/9/18 7:56 PM: - This is a)

[jira] [Commented] (SPARK-21542) Helper functions for custom Python Persistence

2018-11-09 Thread John Bauer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681895#comment-16681895 ] John Bauer commented on SPARK-21542: This is a) much more minimal, b) genuinely useful, and c)

[jira] [Comment Edited] (SPARK-21542) Helper functions for custom Python Persistence

2018-11-09 Thread John Bauer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681895#comment-16681895 ] John Bauer edited comment on SPARK-21542 at 11/9/18 7:54 PM: - This is a)

[jira] [Commented] (SPARK-21542) Helper functions for custom Python Persistence

2018-11-09 Thread John Bauer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681891#comment-16681891 ] John Bauer commented on SPARK-21542: {code} from pyspark.sql import SparkSession from

[jira] [Commented] (SPARK-21542) Helper functions for custom Python Persistence

2018-10-01 Thread John Bauer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16634679#comment-16634679 ] John Bauer commented on SPARK-21542: The above is not as minimal as I would have liked... It is

[jira] [Commented] (SPARK-21542) Helper functions for custom Python Persistence

2018-10-01 Thread John Bauer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16634677#comment-16634677 ] John Bauer commented on SPARK-21542: {code:python} #!/usr/bin/env python3 # -*- coding: utf-8 -*-

[jira] [Commented] (SPARK-21542) Helper functions for custom Python Persistence

2018-09-11 Thread John Bauer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611085#comment-16611085 ] John Bauer commented on SPARK-21542: You don't show your code for __init__ or setParams.  I recall

[jira] [Created] (SPARK-23955) typo in parameter name 'rawPredicition'

2018-04-10 Thread John Bauer (JIRA)
John Bauer created SPARK-23955: -- Summary: typo in parameter name 'rawPredicition' Key: SPARK-23955 URL: https://issues.apache.org/jira/browse/SPARK-23955 Project: Spark Issue Type: Bug