prodeezy commented on issue #4666:
URL: https://github.com/apache/iceberg/issues/4666#issuecomment-1113467212
@mehtaashish23 Thanks for pointing this out. That is correct. Even with
regular commit flow the snapshot gets promoted to master snapshots line after
abort.
Add a throw new RuntimeException("FATAL") anywhere in this block and build
iceberg :
https://github.com/apache/iceberg/blob/master/core/src/main/java/org/apache/iceberg/SnapshotProducer.java#L328-L346
```
public void testWriteBatchWithFailure() throws IOException {
File parent = temp.newFolder(format);
File location = new File(parent, "test-oom");
HadoopTables tables = new HadoopTables(CONF);
Schema SCHEMA = new Schema(
Types.NestedField.optional(1, "id", Types.IntegerType.get()),
Types.NestedField.optional(2, "data", Types.StringType.get())
);
PartitionSpec spec =
PartitionSpec.builderFor(SCHEMA).identity("data").build();
Map tableProps = new HashMap<String, String>();
tableProps.put("write.format.default", "parquet");
Table table = tables.create(SCHEMA, spec, tableProps,
location.toString());
List<SimpleRecord> records = Lists.newArrayList(
new SimpleRecord(1, "a"),
new SimpleRecord(2, "b"),
new SimpleRecord(3, "c")
);
Dataset<Row> df = spark.createDataFrame(records, SimpleRecord.class);
try {
df.write()
.format("iceberg")
.mode("append")
.save(location.toString());
} finally {
table.refresh();
System.out.println("=> Table current after failure: " +
table.currentSnapshot());
System.out.println("=> Table snapshots: " +
Iterables.toString(table.snapshots()));
spark.read().format("iceberg").load(location.toString()).show();
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
