spark-test commented on code in PR #59573:
URL: https://github.com/apache/spark/pull/59573#discussion_r4309969702
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala:
##########
@@ -156,6 +202,10 @@ trait GeneratePredicateHelper extends PredicateHelper {
notNullPreds: Seq[Expression],
otherPreds: Seq[Expression],
nonNullAttrExprIds: Seq[ExprId]): String = {
+ // The IsNotNull checks are never NULL, so only `otherPreds` can need the
flag.
+ val nullFlag =
+ nullConjunctFlag(ctx, otherPreds.map(BindReferences.bindReference(_,
outputAttrs)))
Review Comment:
**Nit (P3):** This map binds every other predicate against outputAttrs, but
genPredicate binds each one against the same schema again. BindReferences
traverses the expression and initializes an AttributeSeq ordinal map for each
attribute-referencing bind, so the extra pass adds driver CPU and allocation
that scales with both conjunct count and schema width. It runs even when the
legacy setting makes nullConjunctFlag return None. Could we retain these bound
expressions for predicate emission, pairing them with the originals needed for
reference and null-check interleaving? This concerns code generation overhead
in the helper, including aggregate FILTER, rather than per-row evaluation or
CSE reuse.
**Verification:**
- **Inspection:** Inspect preparation and emission to verify the same bound
predicate is used for nullable inspection and body generation, with no second
per-predicate output-schema bind.
- **Compatibility:** Compare generated predicate semantics and output/error
behavior for legacy true/false and nullable/nonnullable conditions, including
aggregate FILTER; CSE route behavior remains unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
