Github user maryannxue commented on a diff in the pull request:
https://github.com/apache/spark/pull/22259#discussion_r224510115
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala
---
@@ -47,7 +48,8 @@ case class ScalaUDF(
inputTypes: Seq[DataType] = Nil,
udfName: Option[String] = None,
nullable: Boolean = true,
- udfDeterministic: Boolean = true)
+ udfDeterministic: Boolean = true,
+ nullableTypes: Seq[Boolean] = Nil)
--- End diff --
Yeah, understood and agree. We'd probably need to change 3 things in
follow-up PR(s):
1. Add `isInstanceOf[KnownNotNull]` back (I can do this).
2. Move `nullableTypes` up in the parameter list and make it required. I
agree that it'd only make things worse if anyone uses this private API and
forget to set this parameter.
3. Flip the `true`/`false` around for `nullableTypes`. This is rather
minor, but helps with readability and consistency I think.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
