JunRuiLee opened a new issue, #564: URL: https://github.com/apache/paimon-rust/issues/564
### Background `DataSplit` already supports serialization to the Java-native wire format — `DataSplit::serialize()` (raw v8, byte-compatible with Java `DataSplit#serialize`) and `DataSplit::serialize_split_v1()` (the `SplitSerializer` v1 frame), both golden-verified. There is currently **no** inverse: paimon-rust cannot reconstruct a `DataSplit` from bytes. This blocks the split-planning-vs-reading-separation pattern where paimon-rust is the **reader**: a planner in another process/engine (e.g. a Java frontend) serializes a `DataSplit` and ships it to a worker, which must deserialize it before reading. Today that direction is unavailable, so paimon-rust cannot serve as a distributed read kernel behind an external planner. ### Proposal Add the inverse of the existing serialization, built on existing primitives (no new low-level infra): - **core**: `DataFileMeta::from_serialized_row_data`, `DataSplit::deserialize` (raw v8), `DataSplit::deserialize_split_v1` (`SplitSerializer` frame, DataSplit + IndexedSplit), plus the leaf decoders (binary arrays, SimpleStats, Java modified-UTF, deletion-file list). - **C binding**: `paimon_plan_from_split_bytes` — build a one-split plan from raw-v8 `DataSplit` bytes, so C consumers (engine integrations) can read splits planned elsewhere. Round-trip byte-compatible with the existing serialize; malformed input yields typed errors and never panics; whole-buffer consumption enforced. v8 only for now (current Java `DataSplit.VERSION`), structured so a future v9 branch is a localized addition. The Python binding (`Split.deserialize`) is intentionally out of scope here and can follow separately if pypaimon needs to read splits planned by another implementation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
