JingsongLi commented on code in PR #985:
URL: https://github.com/apache/paimon-rust/pull/985#discussion_r4172479407
##########
crates/paimon/src/table/format_partition.rs:
##########
@@ -309,7 +315,11 @@ pub fn format_partition_value(
(Datum::Int(value), DataType::Int(_)) => Some(value.to_string()),
(Datum::Long(value), DataType::BigInt(_)) => Some(value.to_string()),
(Datum::String(value), DataType::Char(_) | DataType::VarChar(_)) => {
- if value.trim().is_empty() {
+ // Fold to the default partition name exactly when Java
+ // `InternalRowPartitionComputer` does (`isNullOrWhitespaceOnly`),
so
+ // the directory matches cross-engine; `str::trim` uses a different
+ // whitespace set (folds NBSP, keeps U+001C-U+001F).
+ if is_java_whitespace_only(value) {
Review Comment:
[P1] Align partition mutation validation before folding Java-only blanks
This new fold maps U+001C–U+001F to the default partition, but
`parse_format_partition_spec` in DataFusion still validates mutating string
values with `text.trim().is_empty()` (format_partition_ddl.rs:493). Rust trim
keeps these controls, so `ALTER TABLE ... DROP PARTITION (label = '<raw
U+001C>')` is accepted and now selects the existing NULL/default partition. A
real SQL probe with a registered NULL partition containing a Parquet file
returns success, removes its registration, and deletes its directory. Restoring
only this formatter file to the merge-base makes the same probe pass: the NULL
registration/file remain and the literal partition is reported absent. The Java
implementation deliberately rejects these ambiguous mutations before they can
address NULL data. Please update the mutating-value guard to the same
modern-Java whitespace classification and add a DROP regression asserting no
registry/filesystem changes for these characters. Reference:
[PaimonFormatTable.requireN
ameablePartitionValues](https://apache.googlesource.com/paimon/+/ea306e2b2ca5ad82c9b61a2ef8c46eccaf626285/paimon-spark/paimon-spark-common/src/main/scala/org/apache/paimon/spark/format/PaimonFormatTable.scala#241).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
