FounderHy opened a new issue #3172:
URL: https://github.com/apache/iceberg/issues/3172
Iceberg version: 0.12.0
I use `DELETE FROM` sql to delete data from iceberg table, the sql like this:
```sql
DELETE FROM order WHERE pay_date between '2021-09-12' and '2021-09-13'
```
I execute this sql through spark, and found the result of `canDeleteWhere`
is false
After check the code, I found this, the field in the filter must be identity
partition, I want to know how to use the delete sql in iceberg
```java
@Override
public boolean canDeleteWhere(Filter[] filters) {
if (table().specs().size() > 1) {
// cannot guarantee a metadata delete will be successful if we have
multiple specs
return false;
}
Set<Integer> identitySourceIds = table().spec().identitySourceIds();
Schema schema = table().schema();
for (Filter filter : filters) {
// return false if the filter requires rewrite or if we cannot
translate the filter
if (requiresRewrite(filter, schema, identitySourceIds) ||
SparkFilters.convert(filter) == null) {
return false;
}
}
return true;
}
private boolean requiresRewrite(Filter filter, Schema schema, Set<Integer>
identitySourceIds) {
// TODO: handle dots correctly via v2references
// TODO: detect more cases that don't require rewrites
Set<String> filterRefs = Sets.newHashSet(filter.references());
return filterRefs.stream().anyMatch(ref -> {
Types.NestedField field = schema.findField(ref);
ValidationException.check(field != null, "Cannot find field %s in
schema", ref);
return !identitySourceIds.contains(field.fieldId());
});
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
