hcrosse opened a new pull request, #2673: URL: https://github.com/apache/iceberg-go/pull/2673
## What `makePositionDeleteRecordsForFilter` read the position delete files and started its scan workers before returning its sequence. When a delete failed before reading that sequence, the workers kept running and the position deletes were not released until the caller's context ended. A failed deletion vector read does this, and so does a writer that rejects an unknown partition transform. On main, five deletes that failed on an unknown partition transform left 10 scan goroutines and 5 sequencer goroutines running. The returned sequence now reads the delete files and starts its workers only when iterated. `recordBatchesWithCleanup` starts its workers the same way: https://github.com/apache/iceberg-go/blob/74b684cf085355bfddba0d2c74743bacd5d2eae3/table/arrow_scanner.go#L3250-L3261 The writer has several checks that can return before it reads the sequence, so a lazy start covers all of them. Errors from reading delete files are now yielded instead of returned. Successful deletes write the same position deletes as before. ## Testing I added regression tests covering failed deletion vector reads and writer transform errors (both fail on main and pass with this change) plus a test confirming an unreadable position delete file still fails the delete now that the error is yielded. `go test -race ./table/...` and `make lint` pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
