[jira] [Updated] (SPARK-30762) Add dtype="float32" support to vector_to_array UDF
[ https://issues.apache.org/jira/browse/SPARK-30762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-30762: --- Fix Version/s: 3.1.0 3.0.0 > Add dtype="float32" support to vector_to_array UDF > -- > > Key: SPARK-30762 > URL: https://issues.apache.org/jira/browse/SPARK-30762 > Project: Spark > Issue Type: Story > Components: MLlib, PySpark >Affects Versions: 3.0.0 >Reporter: Liang Zhang >Assignee: Liang Zhang >Priority: Major > Fix For: 3.0.0, 3.1.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > Previous PR: > [https://github.com/apache/spark/blob/master/python/pyspark/ml/functions.py] > In the previous PR, we introduced a UDF to convert a column of MLlib Vecters > to a column of lists in python (Seq in scala). Currently, all the floating > numbers in a vector is converted to Double in scala. In this issue, we will > add a parameter in the python function {{vector_to_array(col)}} that allows > converting to Float (32bits) in scala, which would be mapped to a numpy array > of dtype=float32. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-30762) Add dtype="float32" support to vector_to_array UDF
[ https://issues.apache.org/jira/browse/SPARK-30762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-30762: --- Fix Version/s: (was: 3.1.0) (was: 3.0.0) > Add dtype="float32" support to vector_to_array UDF > -- > > Key: SPARK-30762 > URL: https://issues.apache.org/jira/browse/SPARK-30762 > Project: Spark > Issue Type: Story > Components: MLlib, PySpark >Affects Versions: 3.0.0 >Reporter: Liang Zhang >Assignee: Liang Zhang >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > > Previous PR: > [https://github.com/apache/spark/blob/master/python/pyspark/ml/functions.py] > In the previous PR, we introduced a UDF to convert a column of MLlib Vecters > to a column of lists in python (Seq in scala). Currently, all the floating > numbers in a vector is converted to Double in scala. In this issue, we will > add a parameter in the python function {{vector_to_array(col)}} that allows > converting to Float (32bits) in scala, which would be mapped to a numpy array > of dtype=float32. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-30762) Add dtype="float32" support to vector_to_array UDF
[ https://issues.apache.org/jira/browse/SPARK-30762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-30762: --- Fix Version/s: 3.1.0 3.0.0 > Add dtype="float32" support to vector_to_array UDF > -- > > Key: SPARK-30762 > URL: https://issues.apache.org/jira/browse/SPARK-30762 > Project: Spark > Issue Type: Story > Components: MLlib, PySpark >Affects Versions: 3.0.0 >Reporter: Liang Zhang >Assignee: Liang Zhang >Priority: Major > Fix For: 3.0.0, 3.1.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > Previous PR: > [https://github.com/apache/spark/blob/master/python/pyspark/ml/functions.py] > In the previous PR, we introduced a UDF to convert a column of MLlib Vecters > to a column of lists in python (Seq in scala). Currently, all the floating > numbers in a vector is converted to Double in scala. In this issue, we will > add a parameter in the python function {{vector_to_array(col)}} that allows > converting to Float (32bits) in scala, which would be mapped to a numpy array > of dtype=float32. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-30762) Add dtype="float32" support to vector_to_array UDF
[ https://issues.apache.org/jira/browse/SPARK-30762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Zhang updated SPARK-30762: Description: Previous PR: [https://github.com/apache/spark/blob/master/python/pyspark/ml/functions.py] In the previous PR, we introduced a UDF to convert a column of MLlib Vecters to a column of lists in python (Seq in scala). Currently, all the floating numbers in a vector is converted to Double in scala. In this issue, we will add a parameter in the python function {{vector_to_array(col)}} that allows converting to Float (32bits) in scala, which would be mapped to a numpy array of dtype=float32. was: Previous PR: [https://github.com/apache/spark/blob/master/python/pyspark/ml/functions.py] > Add dtype="float32" support to vector_to_array UDF > -- > > Key: SPARK-30762 > URL: https://issues.apache.org/jira/browse/SPARK-30762 > Project: Spark > Issue Type: Story > Components: MLlib, PySpark >Affects Versions: 3.0.0 >Reporter: Liang Zhang >Assignee: Liang Zhang >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > > Previous PR: > [https://github.com/apache/spark/blob/master/python/pyspark/ml/functions.py] > In the previous PR, we introduced a UDF to convert a column of MLlib Vecters > to a column of lists in python (Seq in scala). Currently, all the floating > numbers in a vector is converted to Double in scala. In this issue, we will > add a parameter in the python function {{vector_to_array(col)}} that allows > converting to Float (32bits) in scala, which would be mapped to a numpy array > of dtype=float32. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-30762) Add dtype="float32" support to vector_to_array UDF
[ https://issues.apache.org/jira/browse/SPARK-30762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-30762: -- Component/s: PySpark > Add dtype="float32" support to vector_to_array UDF > -- > > Key: SPARK-30762 > URL: https://issues.apache.org/jira/browse/SPARK-30762 > Project: Spark > Issue Type: Story > Components: MLlib, PySpark >Affects Versions: 3.0.0 >Reporter: Liang Zhang >Assignee: Liang Zhang >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > > Previous PR: > [https://github.com/apache/spark/blob/master/python/pyspark/ml/functions.py] > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org