sam-1112 opened a new pull request, #6111:
URL: https://github.com/apache/datafusion-comet/pull/6111
## Which issue does this PR close?
Closes part of #5646.
## Rationale
Add coverage for a native Iceberg writer failure after data files have
already been finalized.
The test injects a `file://` write failure during a native write, verifies
Spark retries the failed task, and
confirms that the failed attempt leaves no committed or orphaned data
files.
## What changes are included in this PR?
- Add a native Iceberg mid-write failure and retry test.
- Force a native data-file write failure after earlier files are finalized.
- Verify the retry succeeds and produces exactly one Iceberg snapshot.
- Verify the physical data files exactly match Iceberg's referenced files.
- Verify files from the failed attempt are neither present in storage nor
referenced by manifests.
- Verify the final table contains the expected rows without duplicates.
## How was this tested?
```bash
make core
./mvnw test -Pspark-3.5 -Dtest=none \
-Dsuites="org.apache.comet.CometIcebergWriteActionSuite mid-write
failure retries"
./mvnw test -Dtest=none \
-Dsuites="org.apache.comet.CometIcebergWriteActionSuite mid-write
failure retries"
./mvnw spotless:check -Pspark-3.5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
