Yicong Huang created SPARK-60357:
------------------------------------
Summary: Return a named record from read_single_udf for every eval
type
Key: SPARK-60357
URL: https://issues.apache.org/jira/browse/SPARK-60357
Project: Spark
Issue Type: Sub-task
Components: PySpark
Affects Versions: 5.0.0
Reporter: Yicong Huang
read_single_udf ends with an if/elif chain over eval types that packs each UDF
into one of several positional tuple shapes, padded with None placeholders, for
the handler that consumes it. Since each tuple is only a private contract
between read_single_udf and one handler, read_single_udf should instead return
a single named record (and read_single_udtf a UDTF record) for every eval type,
and each EvalTypeHandler should take the fields it needs by name, deriving
per-handler values such as num_udf_args or the kwargs/toInternal wrapping in
its own __init__.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
