RussellSpitzer commented on PR #4692: URL: https://github.com/apache/iceberg/pull/4692#issuecomment-1130513986
I'm a little worried about the instance in which every file was written with the correct sort order BUT were written by independent writes. In this case I have dozens of files which overlap, but they all have the same sort order. In this case I'm not sure it makes sense to ignore the sort request, in this case it wouldn't be redundant and we would be better off if we apply the distribution. For disabling the sort I think disabling the distribution mode is good enough? Although maybe file as task is a nice bonus there? I'm a little worried since that parameter is a bit of a confusing internal implementation detail for an external user to wrap their head around. I am strongly in support though of passing through "sort order" to files and want that PR if we can do it soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
