Takuya Ueshin created SPARK-44479:
-------------------------------------
Summary: Support Python UDTFs with empty schema
Key: SPARK-44479
URL: https://issues.apache.org/jira/browse/SPARK-44479
Project: Spark
Issue Type: Sub-task
Components: PySpark
Affects Versions: 3.5.0
Reporter: Takuya Ueshin
Support UDTFs with empty schema, for example:
{code:python}
>>> class TestUDTF:
... def eval(self):
... yield tuple()
{code}
Currently it fails with `useArrow=True`:
{code:python}
>>> udtf(TestUDTF, returnType=StructType())().collect()
Traceback (most recent call last):
...
ValueError: not enough values to unpack (expected 2, got 0)
{code}
whereas without Arrow:
{code:python}
>>> udtf(TestUDTF, returnType=StructType(), useArrow=False)().collect()
[Row()]
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
