hcrosse opened a new pull request, #2672:
URL: https://github.com/apache/iceberg-go/pull/2672

   ## What
   
   A failed rolling writer removes itself from the factory. `closeAll` listed 
only the writers still present, so it missed the error if the writer had left 
and no fanout worker observed it through `Add`. A partitioned `Append` could 
return nil and commit without that writer's rows.
   
   The factory now keeps every writer created by 
`getOrCreateRollingDataWriter`. `closeAll` reads errors from that list, and 
`abortAll` joins writers that have left the factory. Failed writers still leave 
the partition map, as #1305 intended, so later batches can use a new writer.
   
   This covers writes to partitioned tables through `Append`, `Overwrite`, copy 
on write deletes and `WriteRecords` unless clustered, as well as partitioned v2 
position deletes and partitioned equality deletes.
   
   ## Testing
   
   `TestFanoutWriter/TestWriteReportsFailedUploadAfterWriterLeftFactory` and 
`TestPositionDeletePartitionedFanoutWriterReportsFailedUploadAfterWriterLeftFactory`
 both fail on main and pass with this change. Each makes one file upload fail 
and waits for the writer to leave the factory before the input ends.
   
   With this change, the issue's reproduction returns `error in rolling data 
writer: upload failed` and commits nothing.
   
   `go test -race ./table/...` and `make lint` pass.
   
   Fixes #2671
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to