Malthe Borch created SPARK-53587:
------------------------------------
Summary: Apply SQL-defined function on scalar
Key: SPARK-53587
URL: https://issues.apache.org/jira/browse/SPARK-53587
Project: Spark
Issue Type: Wish
Components: PySpark
Affects Versions: 4.0.0
Reporter: Malthe Borch
While
[transform|https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.transform.html]
applies a function to an array, there is no such functionality for a scalar
value, e.g.
{code:python}
expr = expr.apply(lambda x: x * 2)
{code}
Note that in addition to accepting a Python-function (which is converted into a
Catalyst-expression through introspection), it would be ideal if a SQL-function
(as a string) could be specified as well:
{code:python}
expr = expr.apply("x -> x + 1")
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
