szehon-ho commented on issue #4346: URL: https://github.com/apache/iceberg/issues/4346#issuecomment-1099651817
> That does not solve one case when buckets are indeed different but the path is the same and we have a valid orphan file. Sorry I'm missing this example, why does equalAuthority=false not handle this case and we need more than two flags? One thought to keep the prefix-mismatch-mode flag simple with only error/ignore, is have separate PrefixComparator class: -equals(String thisScheme, String thatScheme, String thisAuth, String thatAuth) with built in S3/HDFS defaults, which can be pluggable or configured, similar like Catalog interface. It would normally not need configuration from user but could allow user to customize it (say restrict HDFS to specific authority equality depending on namenode mappings). But up to you, if this is overkill, or if user needs to frequently customize this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
