Github user maryannxue commented on a diff in the pull request:
https://github.com/apache/spark/pull/22732#discussion_r225605581
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/expressions/UserDefinedFunction.scala
---
@@ -73,27 +73,27 @@ case class UserDefinedFunction protected[sql] (
*/
@scala.annotation.varargs
def apply(exprs: Column*): Column = {
- if (inputTypes.isDefined && nullableTypes.isDefined) {
- require(inputTypes.get.length == nullableTypes.get.length)
+ if (inputTypes.isDefined && handleNullForInputs.isDefined) {
+ require(inputTypes.get.length == handleNullForInputs.get.length)
}
Column(ScalaUDF(
f,
dataType,
exprs.map(_.expr),
+ handleNullForInputs.getOrElse(exprs.map(_ => false)),
--- End diff --
I actually assume the default behavior should be the other way around. If
we don't know, we just do the if-else null handling and it wouldn't do us any
harm correctness-wise, right? Anyway I'm not gonna change that in this PR but
hope we can get an idea if and when this default behavior will happen.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
