[jira] [Commented] (SPARK-31776) Literal lit() supports lists and numpy arrays
[ https://issues.apache.org/jira/browse/SPARK-31776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17626844#comment-17626844 ] Xinrong Meng commented on SPARK-31776: -- `lit` supports Python list and NumPy arrays in https://issues.apache.org/jira/browse/SPARK-39405 in Spark 3.4.0. > Literal lit() supports lists and numpy arrays > - > > Key: SPARK-31776 > URL: https://issues.apache.org/jira/browse/SPARK-31776 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.1.0 >Reporter: Xiangrui Meng >Priority: Major > > In ML workload, it is common to replace null feature vectors with some > default value. However, lit() does not support Python list and numpy arrays > at input. Users cannot simply use fillna() to get the job done. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-31776) Literal lit() supports lists and numpy arrays
[ https://issues.apache.org/jira/browse/SPARK-31776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177281#comment-17177281 ] Zirui Xu commented on SPARK-31776: -- Hi [~mengxr], is this ticket suggesting adding support for Python list and numpy arrays in both lit() and fillna()? I understand that fillna() does not support Array type so adding Python list support to fillna() might involve adding Array support from the SQL side first. > Literal lit() supports lists and numpy arrays > - > > Key: SPARK-31776 > URL: https://issues.apache.org/jira/browse/SPARK-31776 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.1.0 >Reporter: Xiangrui Meng >Priority: Major > > In ML workload, it is common to replace null feature vectors with some > default value. However, lit() does not support Python list and numpy arrays > at input. Users cannot simply use fillna() to get the job done. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
