xumingming commented on PR #59402:
URL: https://github.com/apache/spark/pull/59402#issuecomment-6194201223
Thanks for the thorough review, @HyukjinKwon!
The P2 finding is addressed in d22c01a ([SPARK-56898][SQL] Skip subquery
filters in guard):
1. The union-of-filters guard now excludes any filter containing a subquery
expression (PLAN_EXPRESSION), so IN / NOT IN / EXISTS filters keep the raw
Expand key and a single ExistenceJoin, matching the merge-base plan.
2. Normalization now also skips a conditional count whose condition contains
a subquery, so those counts stay in their own group instead of collapsing
into one that cannot be guarded.
3. Added existence-join-count parity tests (conf on vs off) covering EXISTS,
IN, and NOT IN filters and IF/CASE subquery conditions, plus plan-shape
tests in RewriteDistinctAggregatesSuite.
Both PR description suggestions are applied: the ANSI paragraph now says
COUNT(DISTINCT 10 / x) FILTER (WHERE x <> 0) still raises DIVIDE_BY_ZERO as
before, with the evaluation-domain claim narrowed to unconditionally-safe
children; and the SPARK-58501 bullet now says only the test comment changed,
and why.
Verified locally: RewriteDistinctAggregatesSuite (34 tests),
RewriteDistinctAggregatesConditionalQuerySuite (16 tests), the updated
WholeStageCodegenSuite test, and scalastyle all pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
