hcrosse opened a new pull request, #2672: URL: https://github.com/apache/iceberg-go/pull/2672
## What A failed rolling writer removes itself from the factory. `closeAll` listed only the writers still present, so it missed the error if the writer had left and no fanout worker observed it through `Add`. A partitioned `Append` could return nil and commit without that writer's rows. The factory now keeps every writer created by `getOrCreateRollingDataWriter`. `closeAll` reads errors from that list, and `abortAll` joins writers that have left the factory. Failed writers still leave the partition map, as #1305 intended, so later batches can use a new writer. This covers writes to partitioned tables through `Append`, `Overwrite`, copy on write deletes and `WriteRecords` unless clustered, as well as partitioned v2 position deletes and partitioned equality deletes. ## Testing `TestFanoutWriter/TestWriteReportsFailedUploadAfterWriterLeftFactory` and `TestPositionDeletePartitionedFanoutWriterReportsFailedUploadAfterWriterLeftFactory` both fail on main and pass with this change. Each makes one file upload fail and waits for the writer to leave the factory before the input ends. With this change, the issue's reproduction returns `error in rolling data writer: upload failed` and commits nothing. `go test -race ./table/...` and `make lint` pass. Fixes #2671 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
