uros-b commented on code in PR #56636:
URL: https://github.com/apache/spark/pull/56636#discussion_r3448557776
##########
sql/core/src/test/scala/org/apache/spark/sql/DynamicPartitionPruningSuite.scala:
##########
@@ -1801,140 +1797,6 @@ abstract class DynamicPartitionPruningV1Suite extends
DynamicPartitionPruningDat
}
}
- test("DPP requires every leaf of a materialized filtering side to be
materialized") {
Review Comment:
The PR removes two tests and adds none. The stated motivation is that the
allowlist "over-rejects legitimate deterministic materialized sides (e.g. an
aggregate, or any non-allowlisted operator, over a materialized input)." That
now-enabled case is not locked in by any positive test, so nothing prevents a
future change from silently re-narrowing it. Consider adding one small
assertion that a deterministic non-allowlisted operator over a checkpoint (e.g.
checkpoint...groupBy(...).agg(...)) does produce a standalone DPP subquery.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
