szehon-ho edited a comment on pull request #2925: URL: https://github.com/apache/iceberg/pull/2925#issuecomment-1000688746
This change now adds validation for conflicting delta files. Implementation note: One option was to convert PartitionSet => Expression to re-use existing validation methods that take Expression. But as per earlier comment, we go with new version of validateAddedFiles that takes in PartitionSet, we also now add new version of validateNoNewDeletesForDataFiles that takes PartitionSet. On UX side, we have this enabled by user by setting following Spark write conf: validate-from-snapshot, as I understood from comment: https://github.com/apache/iceberg/pull/3069#discussion_r711803863. This then triggers validation if dynamic.overwrite.isolation.level is serializable (the new default), else there is no validation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
