Github user maryannxue commented on a diff in the pull request:
https://github.com/apache/spark/pull/22732#discussion_r225762708
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/expressions/UserDefinedFunction.scala
---
@@ -81,11 +81,11 @@ case class UserDefinedFunction protected[sql] (
f,
dataType,
exprs.map(_.expr),
+ nullableTypes.map(_.map(!_)).getOrElse(exprs.map(_ => false)),
--- End diff --
In addition to what I just pointed out, which is when we did try to get
`inputSchemas` through `ScalaReflection.schemaFor` and got an exception for
unrecognized types, there's another case where we could get an unspecified
`nullableTypes`, and that is when `UserDefinedFunction` is instantiated calling
the constructor but not the `create` method.
Then I assume it's created by an earlier version, and we should use the old
logic, i.e., `ScalaReflection.getParameterTypes`
(https://github.com/apache/spark/pull/22259/files#diff-57b3d87be744b7d79a9beacf8e5e5eb2L2153)
to get the correct information for `nullableTypes`. Is that right, @cloud-fan
@srowen ?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
