[
https://issues.apache.org/jira/browse/SPARK-43967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-43967:
---------------------------------
Description:
Support UDTFs with empty returns, for example:
{code:java}
@udtf(returnType="a: int")
class TestUDTF:
def eval(self, a: int):
... {code}
Currently, this will fail with the exceptionÂ
{code:java}
TypeError: 'NoneType' object is not iterable {code}
Another example
{code:java}
class TestUDTF:
def eval(self, a: int):
yield {code}
This will fail with the exceptions
{code:java}
java.lang.NullPointerException {code}
Note, arrow-optimized UDTFs already support this. This error only occurs with
regular Python UDTFs.
was:
Support UDTFs with empty returns, for example:
{code:java}
@udtf(returnType="a: int")
class TestUDTF:
def eval(self, a: int):
... {code}
Currently, this will fail with the exceptionÂ
{code:java}
TypeError: 'NoneType' object is not iterable {code}
Another example
{code:java}
class TestUDTF:
def eval(self, a: int):
yield {code}
This will fail with the exceptions
{code:java}
java.lang.NullPointerException {code}
> Support Python UDTFs with empty return values
> ---------------------------------------------
>
> Key: SPARK-43967
> URL: https://issues.apache.org/jira/browse/SPARK-43967
> Project: Spark
> Issue Type: Sub-task
> Components: PySpark
> Affects Versions: 3.5.0
> Reporter: Allison Wang
> Priority: Major
>
> Support UDTFs with empty returns, for example:
> {code:java}
> @udtf(returnType="a: int")
> class TestUDTF:
> def eval(self, a: int):
> ... {code}
> Currently, this will fail with the exceptionÂ
> {code:java}
> TypeError: 'NoneType' object is not iterable {code}
> Another example
> {code:java}
> class TestUDTF:
> def eval(self, a: int):
> yield {code}
> This will fail with the exceptions
> {code:java}
> java.lang.NullPointerException {code}
> Note, arrow-optimized UDTFs already support this. This error only occurs with
> regular Python UDTFs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
