gene-bordegaray commented on issue #22395: URL: https://github.com/apache/datafusion/issues/22395#issuecomment-5395499808
> [@gene-bordegaray](https://github.com/gene-bordegaray), [@jayshrivastava](https://github.com/jayshrivastava), [@adriangb](https://github.com/adriangb): I've filed two issues/fixes for things I encountered during testing of Range partitioning: > > * In three-way joins where one table is already range partitioned, we'd prefer to re-partition the side which is already in flight to match. > > * [Avoid unnecessary repartitioning when joining against pre-partitioned relations #24599](https://github.com/apache/datafusion/issues/24599) > * [ Adapt existing partitioning for unsatisfied inputs in co-partitioned joins #24600](https://github.com/apache/datafusion/pull/24600) > * When doing the above, Range repartitioning can become a bottleneck. > > * [Improve the performance of range re-partitioning #24597](https://github.com/apache/datafusion/issues/24597) > * [Range partition on primitives. #24598](https://github.com/apache/datafusion/pull/24598) awesome awesome, thank you @stuhood . These will be top priority to review today / tomorrow. I would also like to start a new epic / effort around optimizations and strengthening the Range partitioning behavior in DF just like these are beginning to do. I will plan this and add these to that new effort 🙇 cc: @NGA-TRAN -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
