jiwen624 opened a new pull request, #56860:
URL: https://github.com/apache/spark/pull/56860
### What changes were proposed in this pull request?
Make attribute substitution in
`QueryPlanConstraints.inferAdditionalConstraints` collation-aware. For a
binary-stable `source`, substitution is unchanged (any position). For a
non-binary-stable (e.g. collated) `source`, substitute only where it is a
direct operand of a same-collation comparison, so it is never moved into a
context that changes the effective collation.
This mirrors the existing guard in another rule `ConstantPropagation`
(SPARK-55647).
### Why are the changes needed?
`inferAdditionalConstraints` infers a predicate on `b` from `a = b` and a
predicate on `a` by substituting `a` with `b`. This is only valid when `a` and
`b` are byte-for-byte interchangeable. Under a non-binary-stable collation, `a
= b` is a collation equality (e.g. `'hello' = 'HELLO'` under `UTF8_LCASE`), not
byte equality, so substituting into a comparison evaluated in a different
collation produces a wrong constraint and silently drops rows:
```sql
CREATE TABLE t (a STRING COLLATE UTF8_LCASE, b STRING COLLATE UTF8_LCASE);
INSERT INTO t VALUES ('hello', 'HELLO');
SELECT a, b FROM t WHERE a = b AND a = 'hello' COLLATE UTF8_BINARY;
```
### Does this PR introduce _any_ user-facing change?
Yes, fixed incorrect query result.
### How was this patch tested?
Added UT.
### Was this patch authored or co-authored using generative AI tooling?
Yes. Claude Code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
