Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22375#discussion_r217495874
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelperSuite.scala
---
@@ -35,6 +36,13 @@ class ExpressionEvalHelperSuite extends SparkFunSuite
with ExpressionEvalHelper
val e = intercept[RuntimeException] {
checkEvaluation(BadCodegenExpression(), 10) }
assert(e.getMessage.contains("some_variable"))
}
+
+ test("SPARK-25388: checkEvaluation should fail if nullable in DataType
is incorrect") {
+ val e = intercept[RuntimeException] {
+ checkEvaluation(MapIncorrectDataTypeExpression(), Map(3 -> 7, 6 ->
null))
--- End diff --
I see, so the example above passes in codegen off and fails with codegen on
with this fix, while using `Map(3 -> 7, 6 -> -1)` passes codegen on and fails
codegen off, am I right?
What I am thinking about (but I have not yet found a working
implementation) is: since the problem arise when we say we expect `null` in a
non-nullable datatype, can we add such a check? I mean, instead of pretending
the expected value to be nullable, can't we add a check in case it is not
nullable for being sure that it does not contain `null`? I think it would be
better, because we would be able to distinguish a failure caused by a bad test,
ie. a test written wrongly, from a UT failure caused by a bug in what we are
testing. What do you think?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
