Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18906#discussion_r162857704
--- Diff: python/pyspark/sql/functions.py ---
@@ -2103,6 +2103,14 @@ def udf(f=None, returnType=StringType()):
>>> import random
>>> random_udf = udf(lambda: int(random.random() * 100),
IntegerType()).asNondeterministic()
+ .. note:: The user-defined functions are considered to be able to
return null values by default.
+ If your function is not deterministic, call `asNonNullable` on the
user defined function.
--- End diff --
`nullable` instead of `deterministic`?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
