liangjie3138 opened a new issue, #9425:
URL: https://github.com/apache/paimon/issues/9425

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/paimon/issues) 
and found nothing similar.
   
   
   ### Paimon version
   
   2.1
   
   ### Compute Engine
   
   python
   
   ### Minimal reproduce step
   
    ```python
     import io
     import struct
     import zlib
   
     from pypaimon.table.row.blob import BlobData
     from pypaimon.write.blob_format_writer import BlobFormatWriter
   
     output = io.BytesIO()
     writer = BlobFormatWriter(output)
     writer.add_blob("blob", BlobData(b"hello"))
   
     record = output.getvalue()
   
     # CRC stored by PyPaimon
     actual_crc = struct.unpack("<I", record[-4:])[0]
   
     # Java calculates CRC over the whole record except the CRC field itself.
     expected_crc = zlib.crc32(record[:-4]) & 0xFFFFFFFF
   
     print(actual_crc == expected_crc)
   ```
   
     The same issue also affects ARRAY<BLOB>. MAP<K, BLOB> is already 
consistent with Java.
   
   ### What doesn't meet your expectations?
   
   PyPaimon excludes the 8-byte record length when calculating CRC32 for BLOB 
and ARRAY<BLOB>, while Java Paimon includes it. MAP<K, BLOB> already includes 
the record length and is consistent with Java Paimon.
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to