yeshengm opened a new pull request #25036: [SPARK-28236][SQL] Fix PullOutNondeterministic Analyzer rule to enforce idempotence URL: https://github.com/apache/spark/pull/25036 ## What changes were proposed in this pull request? `PullOutNondeterministic` is a once analyzer rule, therefore should follow idempotence property. The `PullOutNondeterministic` rule transforms aggregates when the aggregating expression has sub-expressions whose `deterministic` field is set to false. However, this might break `PullOutNondeterministic`'s idempotence property since the actually aggregation rewriting will only transform those with `NonDeterministic` trait, leading to useless intermediate projects. ## How was this patch tested? Existing UTs and new UTs.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
