sam-1112 opened a new pull request, #6119:
URL: https://github.com/apache/datafusion-comet/pull/6119
## Which issue does this PR close?
Closes part of #5646 (driver-side Iceberg commit failure cleanup).
## Rationale for this change
When every native write task succeeds but the driver-side Iceberg commit
fails, Comet must abort the complete set of task commit messages. Otherwise,
files produced by the failed write can be left behind as orphans.
## What changes are included in this PR?
- Add a scoped, test-only pre-commit hook to `IcebergCommitExec`.
- Add a native Iceberg DELETE regression test that creates a real
commit-time conflict after all task commit messages have been collected.
- Verify that the original Iceberg conflict reaches the caller, the
concurrent append is the only committed change, and the failed DELETE’s files
are removed from both storage and Iceberg metadata.
- Verify the failure path reaches both `CometIcebergWriteExec` and
`IcebergCommitExec`.
## How are these changes tested?
```bash
make core
./mvnw test -Pspark-3.5 -Dtest=none \
'-Dsuites=org.apache.comet.CometIcebergWriteActionSuite driver commit
failure aborts'
./mvnw test -Dtest=none \
'-Dsuites=org.apache.comet.CometIcebergWriteActionSuite driver commit
failure aborts'
./mvnw spotless:check -Pspark-3.5
```
Before merge, apply the run-iceberg-tests label to validate the Iceberg
version matrix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
