li36909 commented on pull request #30248:
URL: https://github.com/apache/spark/pull/30248#issuecomment-722168240
> Just curious. Why would you run `system.exit`? You should better just
throw a proper exception. Calling `system.exit` inside the UDFs wouldn't
terminate user applications as intended.
Below is the case I encountered, and it's cause by the code had a method
named 'testUdf' and a udf obj also named 'testUdf', then worker exit. And I
think the application should not hangup without any information anyway
```
import pyspark.sql.functions as F
from pyspark.sql.types import *
from pyspark.sql import SparkSession
def testUdf(x):
return x+1
spark = SparkSession.builder.appName("test").getOrCreate()
testUdf=F.udf(lambda x: testUdf(x),DoubleType())
df = spark.sql("select float(2.145025959889993) as a")
df.withColumn("test", testUdf(df.a)).show()
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
