aokolnychyi commented on PR #4692: URL: https://github.com/apache/iceberg/pull/4692#issuecomment-1130549098
> I'm a little worried about the instance in which every file was written with the correct sort order BUT were written by independent writes. In this case I have dozens of files which overlap, but they all have the same sort order. In this case I'm not sure it makes sense to ignore the sort request, in this case it wouldn't be redundant and we would be better off if we apply the distribution. I guess the answer would be "it depends". In case of @SinghAsDev, the files seem to be properly compacted and sorted so that only a small number of files overlap. Anyway, the point I was trying to make is there are lot of assumptions that must be met that makes this use case pretty narrow. On top of that, we can't do that at all as Iceberg doesn't know whether a broadcast join will be used. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
