mahdibh opened a new issue, #8642:
URL: https://github.com/apache/iceberg/issues/8642
### Apache Iceberg version
1.2.0
### Query engine
None
### Please describe the bug 🐞
I can specify my own delete handler when using a DeleteFiles operation
directly (through `table.newDelete()`). However, when doing it through a
transaction (ie, `table.newTransaction().newDelete()`) it fails with the
following exception
```
java.lang.IllegalArgumentException: Cannot set delete callback more than
once
at
org.apache.iceberg.relocated.com.google.common.base.Preconditions.checkArgument(Preconditions.java:145)
at
org.apache.iceberg.SnapshotProducer.deleteWith(SnapshotProducer.java:164)
at IcebergDeleteTest.main(IcebergDeleteTest.java:42)
```
I would expect the delete to work the same way within a transaction. If the
internal code needs to do cleanups, those should be handled separately from the
actual cleanups that happen as a result of the delete operation.
Here is a full repro with the successful case first, then the failed one
```
import org.apache.hadoop.conf.Configuration;
import org.apache.iceberg.DeleteFiles;
import org.apache.iceberg.Schema;
import org.apache.iceberg.Table;
import org.apache.iceberg.Transaction;
import org.apache.iceberg.catalog.Catalog;
import org.apache.iceberg.catalog.TableIdentifier;
import org.apache.iceberg.hadoop.HadoopCatalog;
import org.apache.iceberg.types.Types;
public class IcebergDeleteTest {
public static void main(String[] args) {
// Specify the local directory where the Hadoop catalog will store
metadata
String catalogPath = "file:///tmp/catalog";
// Create a Hadoop Catalog
Catalog catalog = new HadoopCatalog(new Configuration(), catalogPath);
// Define the table identifier
TableIdentifier tableIdentifier = TableIdentifier.of("my_database",
"my_table");
// Create a schema for your data
Schema schema = new Schema(Types.NestedField.optional(1, "id",
Types.IntegerType.get()));
// Create a table
Table table = catalog.createTable(tableIdentifier, schema);
// delete directly (works fine)
DeleteFiles deleteFiles = table.newDelete();
deleteFiles
.deleteWith(
(path) -> {
System.err.println("deleting file " + path);
})
.deleteFile("file://foo1")
.commit();
// delete through a transaction (throws exception)
Transaction transaction = table.newTransaction();
deleteFiles = transaction.newDelete();
deleteFiles
.deleteWith(
(path) -> {
System.err.println("deleting file " + path);
})
.deleteFile("file://foo2")
.commit();
transaction.commitTransaction();
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
