allisonwang-db commented on code in PR #48074:
URL: https://github.com/apache/spark/pull/48074#discussion_r1755241144
##########
python/pyspark/sql/udtf.py:
##########
@@ -376,10 +376,13 @@ def __call__(self, *args: "ColumnOrName", **kwargs:
"ColumnOrName") -> "DataFram
from pyspark.sql.classic.column import _to_java_column, _to_seq
from pyspark.sql import DataFrame, SparkSession
+ from pyspark.sql.functions import lit
spark = SparkSession._getActiveSessionOrCreate()
sc = spark.sparkContext
+ args = tuple(lit(arg) for arg in args)
Review Comment:
Hmm we can support column references with lateral joins for Python UDTFs in
the future, so the `arg` input here might not always be literals (e.g.
`df.join(MyUDTF("c1", "c2"), ...)` where "c1" and "c2" are column references.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
