Dandandan commented on PR #4926: URL: https://github.com/apache/arrow-datafusion/pull/4926#issuecomment-1384326722
The repartitioning isn't really redundant though? Suppose we only read from one parquet file and do repartition by hash after, we would gain parallelism by first doing a (fast) round robin partition, and a (slower) hash repartition which can now be parallelized. The round robin repartitioning however would benefit from being also down the `FilterExec` in this case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
