JingsongLi commented on PR #985:
URL: https://github.com/apache/paimon-rust/pull/985#issuecomment-5934449867
Requirement fit: **SUPPORTED** — partition-path interoperability and correct
pruning have end-to-end value. Implementation: **FINDINGS** at `85b3f917`.
**[P2] Preserve U+180E partitions written by supported modern JVMs**
(`crates/paimon/src/table/format_partition.rs:322`; the native string arm at
`spec/partition_utils.rs:401` has the same new fold). The reused helper
includes U+180E based on its existing JDK 8 binary test. On Java 11/17,
`Character.isWhitespace('\u180E')` is false, and Paimon's
`StringUtils.isNullOrWhitespaceOnly` therefore keeps the literal partition
value. The newly changed formatter converts it to the default partition name.
Equality pruning then selects that default directory and omits the existing
literal partition.
I reproduced this with a real Parquet format table and SQL: an unfiltered
scan of a `dt=<U+180E>/active=true` directory sees its row, but `WHERE dt =
'<U+180E>'` returns no rows at this head. The identical probe returns the row
with all three PR-modified production files restored to merge-base `6267e9c0`.
Local Java 11 and 17 probes both return false for U+180E. Please define the JVM
compatibility target explicitly, preserve reads of the modern-Java literal
layout, and add this SQL regression rather than extending the JDK 8-only
classification to every string path.
Verification: 200/200 partition-filtered core tests passed. The existing
DataFusion partition-filter integration test passed; the added U+180E
regression failed on the PR and both tests passed on the baseline. Diff check
and current-main merge-tree are clean; all 14 head CI checks are green.
Temporary edits were restored.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
