cloud-fan commented on PR #59402:
URL: https://github.com/apache/spark/pull/59402#issuecomment-6248033783
Thanks. I checked these at 4a8ee10. (1) normalizeCountDistinctConditional
now skips a COUNT whose IF/CASE child is also the distinct child of an
aggregate it will not normalize (protectedChildren,
RewriteDistinctAggregates.scala:518-526). The new four-aggregate test pins the
shared IF(b > 1, c, NULL) group and a guarded c group over exactly b > 2 and b
> 3. (2) The non-distinct COUNT test now pairs it with two conditional distinct
counts on one base, so normalization is kept, and it asserts the raw IF child
and no FILTER. (3) The config doc now lists the deterministic, subquery-free
filter requirement and says subquery-bearing conditions are not rewritten. (4)
The config doc and the PR description describe the OptimizeExpand trade-off and
SPARK-60318. These four items look addressed.
<!-- SPARK_DEV_REVIEW_REPLY
{"feedback_id":"conversation:6230948385","thread_id":"conversation:6230948385","verdict_sha256":"83f41d15fc407f91895b569ad4efe962443ea36d189fb6421d1e4ab13efaf618"}
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
