asolimando commented on issue #27600: URL: https://github.com/apache/datafusion/issues/27600#issuecomment-7057438877
Thanks @stuhood, keeping `RangePartitioning` minimal and carrying emptiness in `Statistics` makes sense to me. I did a quick pass on your prototype to validate your proposal, and I noticed two points worth sharing beyond my validation above: 1. I have verified that `estimate_join_statistics` reports a provably empty join as `Inexact(0)` instead of `Exact(0)`, so exchanges that skip `Exact(0)` inputs cannot skip it. This fix does not depend on the partitioning change, I have filed https://github.com/apache/datafusion/issues/27629 to track it. 2. `ElementwiseFromChildren` relies on operators not reading the partition index, as stated in the doc. A test that compares the batch result with the per-partition walk for each operator that opts in would catch a later change that breaks this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
