JingsongLi commented on PR #1009: URL: https://github.com/apache/paimon-rust/pull/1009#issuecomment-5978038131
The new `with_read_type` API follows Java's `withReadType` model. As the next step, please align the read-type JSON produced by both Rust and the paired PyPaimon PR with Java's existing contract, especially the Variant field descriptions. Java's [`VariantMetadataUtils`](https://github.com/apache/paimon/blob/09d859879ee5ff4765a9bd1ae7fe6b16c1188a05/paimon-common/src/main/java/org/apache/paimon/data/variant/VariantMetadataUtils.java#L45) encodes these descriptions as: ```text __VARIANT_METADATA<path>;<failOnError>;<timeZoneId> ``` For example: `__VARIANT_METADATA$.x;false;UTC`. Rust's `build_variant_metadata` and PyPaimon's `_variant_read_type_json` instead emit a JSON payload after the marker. Java currently only parses the semicolon-separated format. I verified that parsing the current Python-generated description with Java's `failOnError` method throws `ArrayIndexOutOfBoundsException`. Please make both producers emit the Java format first, and add interoperability tests for Java-generated read-type JSON consumed by Rust and Python/Rust-generated read-type JSON consumed by Java, checking the path, error flag, and time zone. Rust can retain its JSON-payload parser for backward compatibility if needed, while newly generated descriptions should use the shared Java representation. Please also apply Java's restriction on semicolons in paths consistently. The overall ROW schema can remain JSON; the incompatibility here is the encoding inside each Variant child's `description`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
