Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/23098#discussion_r235199020
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -939,11 +939,9 @@ trait ScalaReflection extends Logging {
* used with a different Scala version.
*/
def getParameterTypeNullability(func: AnyRef): Seq[Boolean] = {
- if (!Properties.versionString.contains("2.11")) {
- logWarning(s"Scala ${Properties.versionString} cannot get type
nullability correctly via " +
- "reflection, thus Spark cannot add proper input null check for
UDF. To avoid this " +
- "problem, use the typed UDF interfaces instead.")
- }
+ logWarning(s"Scala ${Properties.versionString} cannot get type
nullability correctly via " +
--- End diff --
@maryannxue I think you helped work on this part ... if we're only on Scala
2.12 now can we simplify this further?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
