Ron Serruya created SPARK-48555:
-----------------------------------
Summary: Support Column type for several SQL functions in scala
and python
Key: SPARK-48555
URL: https://issues.apache.org/jira/browse/SPARK-48555
Project: Spark
Issue Type: New Feature
Components: Connect, PySpark, Spark Core
Affects Versions: 3.5.1
Reporter: Ron Serruya
Currently, several SQL functions accept both native types and Columns, but only
accept native types in their scala/python APIs:
* array_remove (works in SQL, scala, not in python)
* array_position(works in SQL, scala, not in python)
* map_contains_key (works in SQL, scala, not in python)
* substring (works only in SQL)
For example, this is possible in SQL:
{code:python}
spark.sql("select array_remove(col1, col2) from values(array(1,2,3), 2)")
{code}
{code:python}
df.select(F.array_remove(F.col("col1"), F.col("col2"))
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
