sundapeng opened a new pull request, #11349:
URL: https://github.com/apache/paimon/pull/11349

   ### Purpose
   
   System tables (`t$files`, `t$buckets`, `t$fileindexes`, `t$filekeyranges`) 
render the partition as a single string column and parse a pushed-down 
Equal/In/range literal back into a partition spec with 
`PartitionPredicateHelper.parsePartitionSpec`, which splits on ", " without a 
limit. A partition value that itself contains ", " (e.g. the STRING partition 
`hello, world`) splits into more fields than there are partition keys, the 
parse returns null, and `applyPartitionFilter` reports no results: the query 
silently returns zero rows for a partition that does have files. In the IN 
branch the same mismatch silently drops the affected literals, so those 
partitions vanish from the results. A trailing empty component (`{a, }` for two 
keys) is also lost, because `String.split` without a limit discards trailing 
empty strings.
   
   This splits with `partitionKeys.size()` as the limit, so a value containing 
", " survives when it is the last partition field, and a trailing empty 
component round-trips. Inputs with at most keys-1 separators are parsed 
identically to before, so no working case changes.
   
   The format stays ambiguous with multiple keys and a separator inside a 
non-last value; such filters can still misparse, but they already return an 
empty result today. #10207 previously reported the same round-trip failure and 
proposed degrading to a full scan; this fixes the parse itself so the filter 
keeps being pushed down.
   
   ### Tests
   
   - `PartitionPredicateHelperTest#testParseValueContainingCommaSpace`: single 
key, value `hello, world` — returns null on master, the spec with fix.
   - 
`PartitionPredicateHelperTest#testParseMultiPartitionLastValueContainsComma`: 
two keys, last value contains the separator — null on master, correct spec with 
fix.
   - `PartitionPredicateHelperTest#testParseTrailingEmptyComponent`: two keys, 
trailing empty component `{a, }` — null on master, `pt1=a, pt2=""` with fix.
   - `FilesTableTest#testReadWithPartitionValueContainingSeparator`: end-to-end 
— a table with the STRING partition `hello, world`, `WHERE partition = '{hello, 
world}'` returns its files with the fix and an empty list on master.
   - 
`FilesTableTest#testReadWithMultiKeyPartitionAndInFilterContainingSeparator`: 
two partition keys whose last value contains the separator — Equal returns the 
partition's files (empty on master), and an IN containing both the 
separator-bearing and a plain literal returns both partitions (the 
separator-bearing literal is silently dropped on master).
   
   On master production code the parser tests and both end-to-end assertions 
fail (`Expecting actual not to be null`, empty list `[]` instead of the 
partition's files, and the IN result missing `{a, b, c}-0-0`); with the fix 
`FilesTableTest` (14/14) and `PartitionPredicateHelperTest` (8/8) pass and 
spotless:check is clean.
   
   ### API and Format
   
   No.
   
   ### Documentation
   
   No.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to