anoopj opened a new pull request, #18546:
URL: https://github.com/apache/iceberg/pull/18546
OverwriteFiles takes the overwrite scope two ways: `overwriteByRowFilter`
(the predicate for the rows to replace) and `conflictDetectionFilter` (the
scope the operation scanned, used with explicit file deletes). They are quite
similar but validation
treated them differently. e.g. the added-data check fell back to
alwaysTrue when files were deleted explicitly, while the delete checks used
the conflict filter or the row filter.
Make the two mutually exclusive and use a single filter for every check: the
conflict detection filter if set, else the overwrite row filter, else
alwaysTrue.
I want to call out that this is a Java API behavior change. I did an audit
of the codebase and this should be safe change at least in the commonly used
engines:
Spark already sets just one of them per commit and Flink doesn't use
OverwriteFiles, so no engine path changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
