JingsongLi commented on PR #976: URL: https://github.com/apache/paimon-rust/pull/976#issuecomment-5934999004
[P2] Do not apply the primary/partition VARIANT restriction to hash bucket keys At `crates/paimon/src/spec/schema.rs:1499-1502`, the shared reject closure now prohibits VARIANT bucket keys as well as primary/partition keys. The default append-table bucket route hashes a BinaryRow; it does not require key ordering, and this codebase already encodes VARIANT values into that row. I verified a Parquet append table with `bucket=2`, `bucket-key=v`, and a VARIANT column `v`: on the exact baseline, it writes, commits, scans and reads two original JSON values successfully, with valid bucket IDs. On this head, schema creation fails with `The VARIANT type of bucket key field 'v' is unsupported.` The upgrade also affects existing tables. Loading the persisted baseline schema succeeds, but a filesystem Catalog `alter_table` that only changes the ordinary `id` column's comment now fails with that same bucket-key error. The identical ALTER passes on the baseline. This makes unrelated metadata maintenance unavailable for tables which were working before this change. Please separate bucket restrictions from primary/partition restrictions and retain the supported VARIANT hash-key path. Java's primitive-key validation applies to primary/partition keys; its separate nested bucket check lists ARRAY/MULTISET/MAP/ROW, not VARIANT. I am not claiming Java's full VARIANT write path is supported—the reproduced regression is in the existing Rust write/read and ALTER paths. Validation on head `c7504c1663e51cad3f9ea3efb28e72c06042c1e7`: 127 schema tests passed. Both additional creation and existing-table ALTER scenarios fail on head; full write/commit/read and the ALTER scenario pass with `schema.rs` restored to the exact baseline. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
