xudong963 commented on PR #15683: URL: https://github.com/apache/datafusion/pull/15683#issuecomment-2801054875
> I do think the current behavior is a potential pain point, because sometimes users know their files are ordered and non-overlapping, but can't get DataFusion to avoid the sort. However, this has been the behavior in DataFusion for a long time (at least the 2 years I have been using it), so if we were to change it, I would prefer if we do it in a way that makes this behavior change _very obvious_. How about adding a new config, such as `check_file_group_ordering`, and make it true as default? In this way, we won't change the current behavior. But I'm concerned that if users only wanna one of the tables to disable the check, they need to implement their own table configuration or something else to avoid the config influencing the global. Also cc @alamb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
