laskoviymishka opened a new pull request, #1255: URL: https://github.com/apache/iceberg-go/pull/1255
readDeletes assumed a position delete file's file_path column was always dictionary-encoded with a string value type and cast it straight to *array.Dictionary / *array.String. Files that store file_path as a plain or large string column (PyIceberg, arrow-rs, or any writer with dictionary encoding turned off) made that cast panic on the scan path. Parse the column by layout instead. Plain string, large string, and dictionary variants of both are supported; any other type is rejected with ErrInvalidSchema. Distinct file paths are gathered in Go rather than through compute.Unique, whose kernel concatenated chunked input into a scratch array that was never released. Build each per-path position list with an explicit mask release after the filter, a checked assertion on the filter result, and chunk cleanup on every error return. Tests cover the four string layouts, distinct per-chunk dictionaries, null and unsupported-layout rejection, and the RowDelta round-trip now writes file_path without dictionary encoding. The unit tests run under a checked allocator to pin reference counting. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
