ebyhr opened a new pull request, #3576: URL: https://github.com/apache/iceberg-python/pull/3576
# Rationale for this change deletion-vector-v1 blobs are wrapped in [length:4][magic:4][Roaring64][CRC:4] per the Puffin spec. The reader passed raw blob bytes directly to _deserialize_bitmap, which expects bare Roaring64, causing "Could not deserialize bitmap" on Spark-written tables. Also fix _payload storing puffin[8:] instead of puffin — blob offsets in the footer are file-relative (from byte 0), not relative to byte 8. The two bugs cancelled for single-blob files at offset 4; the fix was only testable once PuffinFile supported multi-blob/compressed files. Also widen PuffinBlobMetadata.type from Literal["deletion-vector-v1"] to str so PuffinFile can parse files containing non-DV blobs without a Pydantic validation error. ## Are these changes tested? Yes. Copied files from https://github.com/apache/iceberg/tree/main/core/src/test/resources/org/apache/iceberg/puffin/v1 ## Are there any user-facing changes? No <!-- In the case of user-facing changes, please add the changelog label. --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
