[
https://issues.apache.org/jira/browse/SPARK-35344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xinrong Meng updated SPARK-35344:
---------------------------------
Description:
Spark (`lit` function defined in `pyspark.sql.functions`) doesn't support
creating a Column out of numpy literal value.
So `lit` function defined in `pyspark.pandas.spark.functions` should be
adjusted in order to support that in pandas-on-Spark.
was:
Aim to achieve {{F}}{{*.*}}{{lit(np}}{{*.*}}{{int64(1)).}}
We can define {{def lit(literal) -> Column: }}under IntegralOps for example.
OR we can define in python/pyspark/pandas/spark/functions.py.
> Support creating a Column of numpy literal value Support creating a Column of
> numpy literal value in pandas-on-Spark
> --------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-35344
> URL: https://issues.apache.org/jira/browse/SPARK-35344
> Project: Spark
> Issue Type: Sub-task
> Components: PySpark
> Affects Versions: 3.2.0
> Reporter: Xinrong Meng
> Priority: Major
>
> Spark (`lit` function defined in `pyspark.sql.functions`) doesn't support
> creating a Column out of numpy literal value.
> So `lit` function defined in `pyspark.pandas.spark.functions` should be
> adjusted in order to support that in pandas-on-Spark.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
