hcrosse opened a new pull request, #2673:
URL: https://github.com/apache/iceberg-go/pull/2673

   ## What
   
   `makePositionDeleteRecordsForFilter` read the position delete files and 
started its scan workers before returning its sequence. When a delete failed 
before reading that sequence, the workers kept running and the position deletes 
were not released until the caller's context ended. A failed deletion vector 
read does this, and so does a writer that rejects an unknown partition 
transform.
   
   On main, five deletes that failed on an unknown partition transform left 10 
scan goroutines and 5 sequencer goroutines running.
   
   The returned sequence now reads the delete files and starts its workers only 
when iterated. `recordBatchesWithCleanup` starts its workers the same way:
   
   
https://github.com/apache/iceberg-go/blob/74b684cf085355bfddba0d2c74743bacd5d2eae3/table/arrow_scanner.go#L3250-L3261
   
   The writer has several checks that can return before it reads the sequence, 
so a lazy start covers all of them. Errors from reading delete files are now 
yielded instead of returned. Successful deletes write the same position deletes 
as before.
   
   ## Testing
   
   I added regression tests covering failed deletion vector reads and writer 
transform errors (both fail on main and pass with this change) plus a test 
confirming an unreadable position delete file still fails the delete now that 
the error is yielded.
   
   `go test -race ./table/...` and `make lint` pass.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to