rank-p commented on PR #27163: URL: https://github.com/apache/datafusion/pull/27163#issuecomment-6851394099
Hi @alamb, thanks a lot for the review, I really appreciate it. I scoped this PR down. The core logic in `metadata.rs` is now: - Get all `sorting_columns` from row_groups (check if they are equal) - Iterate through `row_groups`, check if bounds are sorted. - If bounds are equal --> decide sorting decision with the next column If statistics are unknown the file is labeled as not ordered. This could be the case if null counts are missing. One question from my side: For the snapshot tests, is this how you are usually doing this? Concretely, this tests would fail e.g. if the default plan is changed and instead of `TopK` the plan would use something else. Not sure if this is a problem or an accepted maintenance cost. Sqllogictest can't test this behavior as it can't write a file like this. Which makes me think if this is even worth fixing at all. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
