Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22259#discussion_r214683934
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala
---
@@ -47,7 +48,8 @@ case class ScalaUDF(
inputTypes: Seq[DataType] = Nil,
udfName: Option[String] = None,
nullable: Boolean = true,
- udfDeterministic: Boolean = true)
+ udfDeterministic: Boolean = true,
+ nullableTypes: Seq[Boolean] = Nil)
--- End diff --
The logic here was again that we wanted to avoid changing the binary
signature. I know catalyst is effectively private to Spark, but this wasn't
marked specifically private; I wondered if it would actually affect callers? If
not we can go back and merge it.
Nil is just an empty list; I don't think it's dangerous and it is used
above in `inputTypes`. It is not always set, because it's not always possible
to infer the schema, let alone nullability.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
