yanbinyang opened a new issue, #603: URL: https://github.com/apache/paimon-rust/issues/603
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon-rust/issues) and found nothing similar. ### Please describe the bug 🐞 `BinaryRow::new(0).to_serialized_bytes()` currently emits only the 4-byte big-endian arity: ```text 00 00 00 00 ``` This is a truncated Paimon `BinaryRow`. The Java reference implementation initializes `BinaryRow.EMPTY_ROW` with its 8-byte fixed part, so `SerializationUtils.serializeBinaryRow(BinaryRow.EMPTY_ROW)` emits 12 bytes: ```text 00 00 00 00 00 00 00 00 00 00 00 00 |-- arity --| |-------- fixed part --------| ``` The problem is exposed by the native `DataSplit` codec added in #565. `DataSplit::serialize()` calls `partition.to_serialized_bytes()`, while `DataSplit::deserialize()` calls the strict `BinaryRow::from_serialized_bytes()` decoder. A non-partitioned split built with the commonly used `BinaryRow::new(0)` therefore cannot deserialize its own output: ```rust let split = DataSplit::builder() .with_snapshot(1) .with_partition(BinaryRow::new(0)) .with_bucket(0) .with_bucket_path("file:/tmp/bucket-0".to_string()) .with_total_buckets(1) .with_data_files(vec![]) .build() .unwrap(); let bytes = split.serialize().unwrap(); DataSplit::deserialize(&bytes).unwrap(); ``` The last line fails with: ```text BinaryRow: serialized body too short for arity 0: 0 bytes, need at least 8 ``` The existing history confirms that the serializer, rather than the decoder, is wrong: - #349 established that a Java-compatible empty `BinaryRow` contains the 4-byte arity and the 8-byte null bitmap/fixed part, and introduced `EMPTY_SERIALIZED_ROW`. - #364 correctly made `BinaryRow::from_serialized_bytes()` reject a truncated body. That validation should not be weakened. - #565 added native `DataSplit` deserialization and non-empty Java goldens, but does not cover a split whose partition is `BinaryRow::new(0)`. I also verified the Java side directly with the Maven Central `org.apache.paimon:paimon-bundle:1.4.2` artifact: - `SerializationUtils.serializeBinaryRow(BinaryRow.EMPTY_ROW)` returns 12 bytes. - Java `DataSplit#serialize` / `DataSplit#deserialize` round-trips an empty-partition split. - The Java-generated empty-partition DataSplit is 77 bytes. Expected behavior: serializing the canonical Rust empty row should produce the same 12-byte representation as Java, and an empty-partition native DataSplit should deserialize and reserialize successfully. ### Solution Normalize only the canonical in-memory stub (`arity == 0 && data.is_empty()`) in `BinaryRow::to_serialized_bytes()` by returning the existing `EMPTY_SERIALIZED_ROW`. This keeps the strict decoder from #364 unchanged, preserves normal populated-row serialization, and restores compatibility with Java without changing the DataSplit wire version. Add regression tests for: - the exact 12-byte Java wire representation of `BinaryRow::new(0)`; - an empty-partition DataSplit against bytes generated by Java Paimon 1.4.2; - deserialize and byte-stable reserialize of that split. ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
