Stephen0421 opened a new pull request, #9539: URL: https://github.com/apache/paimon/pull/9539
### Purpose Align PyPaimon's BlobDescriptor wire format with Java, without changing the existing production read loop. - `serialize()` always writes `CURRENT_VERSION` (v2 + magic), matching Java `BlobDescriptor.serialize()`. - `deserialize()` accepts v1/v2 prefixes, trailing bytes, and signed URI length (`<i`, reject negatives). - `parse_if_serialized` is an exact-length parse for v1/v2 (used by writer validation). - `from_descriptor_bytes` is the explicit API for bytes known to be a descriptor. It uses `deserialize()` (Java prefix contract, trailing padding allowed). It is not a detector: garbage that happens to look like a v1 prefix can yield a BlobRef with a nonsense URI. - `from_bytes()` / `is_blob_descriptor` stay v2-magic-only so inline blob payload is not classified as a v1 descriptor. - `blob-descriptor-field` writes (`write()` and `write_row()`) accept exact v1/v2 bytes and reject trailing padding. Version is checked from `data[0]` before parse (`in [1, CURRENT_VERSION]`). Writer validation uses exact wire length instead of a `serialize()` round-trip, because that round-trip would reject legal v1 after `serialize()` became always-v2. - `blob.stored-descriptor-fields` is still not a layout switch. Python master ignored that key and wrote dedicated `.blob` files; a global fallback would mis-parse those tables on a rolling upgrade. The cost is that Java tables which only set the fallback key store inline descriptors, and Python returns those bytes instead of fetching payload. Use `blob-descriptor-field` (column directives already copy the legacy key onto the canonical option). Out of scope (follow-up): production convert readers still call `Blob.from_bytes()`, so historical v1 descriptor columns are stored successfully but not resolved to payload. That path should use `from_descriptor_bytes`. ### Tests - `pypaimon.tests.blob_test.BlobTest` — descriptor serialize/parse, writer v1/v2/trailing/version, `from_bytes(allow_blob_data=False)` error wrapping - `pypaimon.tests.blob_table_test.DedicatedFormatWriterTest.test_legacy_stored_descriptor_fields_keeps_dedicated_blob_layout` - `pypaimon.tests.column_directive_utils_test` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
