slachiewicz opened a new pull request, #3834: URL: https://github.com/apache/iceberg-python/pull/3834
A null `list<struct<...>>` was rebuilt as an empty list, losing the distinction between the two. `test_null_list_and_map` has documented this since #252, asserting the corrupted value with the correct assertion commented out pending apache/arrow#38809. That dependency turns out not to hold for this symptom: `from_arrays` takes a `mask` argument, so carrying the validity bitmap over fixes the null loss on its own. The rebuild itself is still needed for #38809 and is unchanged. The write path is affected too, and there the loss is permanent — the Parquet file pyiceberg produces contains an empty list, so no reader can tell it from a null. The new unit test covers that direction; the integration assertion covers the read side. Closes #3833, which has the full analysis and a standalone reproduction. *This change was created with AI assistance.* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
