RussellSpitzer commented on issue #4666:
URL: https://github.com/apache/iceberg/issues/4666#issuecomment-1113544716
@prodeezy I ran the following test (I modified it so I could run in
IcebergSourceTablesBase)
```
@Test
public void testWriteBatchWithFailure() throws IOException {
TableIdentifier tableIdentifier = TableIdentifier.of("db", "files_test");
Table table = createTable(tableIdentifier, SCHEMA,
PartitionSpec.builderFor(SCHEMA).identity("id").build());
List<SimpleRecord> records = Lists.newArrayList(
new SimpleRecord(1, "a"),
new SimpleRecord(2, "b"),
new SimpleRecord(3, "c")
);
Dataset<Row> df = spark.createDataFrame(records, SimpleRecord.class);
try {
df.write()
.format("iceberg")
.mode("append")
.save(loadLocation(tableIdentifier));
} finally {
table.refresh();
System.out.println("=> Table current after failure: " +
table.currentSnapshot());
System.out.println("=> Table snapshots: " +
Iterables.toString(table.snapshots()));
spark.read().format("iceberg").load(loadLocation(tableIdentifier))
.show(); // No Error Here
}
}
```
Then I put exceptions in the Snapshot producer block
```
diff --git a/core/src/main/java/org/apache/iceberg/SnapshotProducer.java
b/core/src/main/java/org/apache/iceberg/SnapshotProducer.java
index 80afb04e5..9650d336d 100644
--- a/core/src/main/java/org/apache/iceberg/SnapshotProducer.java
+++ b/core/src/main/java/org/apache/iceberg/SnapshotProducer.java
@@ -335,6 +335,7 @@ abstract class SnapshotProducer<ThisT> implements
SnapshotUpdate<ThisT> {
for (String manifestList : manifestLists) {
if (!saved.manifestListLocation().equals(manifestList)) {
deleteFile(manifestList);
+ throw new RuntimeException("Foobar");
}
}
} else {
@@ -342,6 +343,7 @@ abstract class SnapshotProducer<ThisT> implements
SnapshotUpdate<ThisT> {
// consistency problems in refresh. in that case, don't clean up.
LOG.warn("Failed to load committed snapshot, skipping manifest
clean-up");
}
+ throw new RuntimeException("Foobar");
} catch (RuntimeException e) {
LOG.warn("Failed to load committed table metadata, skipping manifest
clean-up", e);
```
This has no errors. Does this need to happen in conjunction with a WAP
still?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
