XiaoHongbo-Hope opened a new issue, #1466:
URL: https://github.com/apache/paimon-rust/issues/1466
### Problem
Rust BLOB materialization rejects intact files written by Python before
apache/paimon#9426 with `Blob entry CRC32 mismatch`. Those writers calculated
CRC32 over magic + payload, excluding the trailing 8-byte entry length. Current
writers include the length. Fixing the writer did not migrate existing files.
A complete single-row legacy file containing `legacy` is:
```
cf114e586c656761637916000000000000002ab4a15b2c0100000001
```
It was generated with `BlobFormatWriter` from the parent of apache/paimon
commit `19a933b7d96658f19a87301b0990919702464fd3`, calling `add_blob("payload",
BlobData(b"legacy"))` and `close()`.
### Expected behavior
Accept both checksum conventions while continuing to reject corrupted
payloads and invalid framing. Compute both checksums with one payload scan.
Also provide an explicit read option to disable CRC verification for controlled
comparisons; retain verification by default and preserve structural validation
when disabled.
Observed against paimon-rust main
`589e77431f666727369e3c89b2944dc42d5bbe08`. The issue affects scalar, array,
and map BLOB entries; it is independent of table row reconstruction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]