XiaoHongbo-Hope opened a new issue, #1466:
URL: https://github.com/apache/paimon-rust/issues/1466

   ### Problem
   
   Rust BLOB materialization rejects intact files written by Python before 
apache/paimon#9426 with `Blob entry CRC32 mismatch`. Those writers calculated 
CRC32 over magic + payload, excluding the trailing 8-byte entry length. Current 
writers include the length. Fixing the writer did not migrate existing files.
   
   A complete single-row legacy file containing `legacy` is:
   
   ```
   cf114e586c656761637916000000000000002ab4a15b2c0100000001
   ```
   
   It was generated with `BlobFormatWriter` from the parent of apache/paimon 
commit `19a933b7d96658f19a87301b0990919702464fd3`, calling `add_blob("payload", 
BlobData(b"legacy"))` and `close()`.
   
   ### Expected behavior
   
   Accept both checksum conventions while continuing to reject corrupted 
payloads and invalid framing. Compute both checksums with one payload scan. 
Also provide an explicit read option to disable CRC verification for controlled 
comparisons; retain verification by default and preserve structural validation 
when disabled.
   
   Observed against paimon-rust main 
`589e77431f666727369e3c89b2944dc42d5bbe08`. The issue affects scalar, array, 
and map BLOB entries; it is independent of table row reconstruction.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to