chenjunjiedada commented on code in PR #4539:
URL: https://github.com/apache/iceberg/pull/4539#discussion_r855755079
##########
data/src/main/java/org/apache/iceberg/data/DeleteFilter.java:
##########
@@ -185,14 +203,35 @@ private CloseableIterable<T>
applyEqDeletes(CloseableIterable<T> records) {
.reduce(Predicate::and)
.orElse(t -> true);
- Filter<T> remainingRowsFilter = new Filter<T>() {
+ Filter<T> remainingRowsFilter = hasMetadataColumnIsDeleted ?
getMarker(remainingRows) : getFilter(remainingRows);
+
+ return remainingRowsFilter.filter(records);
+ }
+
+ private Filter<T> getFilter(Predicate<T> remainingRows) {
+ return new Filter<T>() {
@Override
protected boolean shouldKeep(T item) {
return remainingRows.test(item);
}
};
+ }
- return remainingRowsFilter.filter(records);
+ private Filter<T> getMarker(Predicate<T> remainingRows) {
+ return new Filter<T>() {
+ @Override
+ protected boolean shouldKeep(T item) {
+ if (!remainingRows.test(item)) {
+ // if the row is deleted, set the metadata column to true
+ markRowDeleted(item);
+ }
+ return true; // keep the row even if it is deleted by equality
deletions
+ }
+ };
+ }
+
+ protected void markRowDeleted(T item) {
Review Comment:
The previous PR which read the deleted rows has such implementation, +1 to
use another PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
