MaxGekk commented on code in PR #59573:
URL: https://github.com/apache/spark/pull/59573#discussion_r4322030687
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala:
##########
@@ -215,42 +323,44 @@ trait GeneratePredicateHelper extends PredicateHelper {
if !generatedIsNotNullChecks(idx) && !n.isInstanceOf[Attribute] &&
Review Comment:
Thank you for catching this, @LuciferYang. I reproduced it on the first row
of `withNullConjunctTable(Int.MaxValue)`: with the flag, `cast(s AS INT) IS NOT
NULL` is generated before `coalesce(b, cast(s AS INT)) < 0`, although
`And.eval` never reaches it when that conjunct is FALSE. I will skip the nested
branch, here and in the CSE branch, once a nullable conjunct has been
generated, and add a test for this shape.
Note that the hoisting itself predates this PR. On a row where the first
conjunct is TRUE, e.g. `(0, 1, 0, 1, 'x')`, the same query raises
`CAST_INVALID_INPUT` under codegen with the legacy config on and off, while the
interpreted mode returns no rows, because the check is hoisted before the
second conjunct regardless of the flag. The nested branch should hoist a check
only when it is written before the conjunct, i.e. only the checks with an index
below `precedingNotNulls(k)`. I opened SPARK-60377 for that, to keep this PR
focused on the NULL conjunct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
