Dandandan commented on PR #21580: URL: https://github.com/apache/datafusion/pull/21580#issuecomment-4234559084
I wonder if the ordering should be done _before_ the files / row groups are assigned to partitions? So they are more _globally_ sorted instead of just per partition? It seems now they are sorted *within* each partition, which should help, but perhaps not nearly as much as it would be if all the partitions contain the optimal row groups? This would also help in the case of https://github.com/apache/datafusion/issues/21581 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
