bryndenZh opened a new pull request, #3500:
URL: https://github.com/apache/fluss/pull/3500
## Purpose
Related issue: #3490
For a partitioned table with `table.datalake.enabled = true` and a
non-`STRING`
partition column (`DATE`, `INT`, `TIMESTAMP`, etc.), default union read
returns
each tiered row twice — once from the lake split and once from the Fluss-log
split.
Root cause: `PaimonSplit#partition()` read every partition field via
`BinaryRow.getString(i)` regardless of logical type, producing garbage for
non-`STRING` columns. The lake-side partition name then never matched the
Fluss-side name in `LakeSplitGenerator#generatePartitionTableSplit`, so the
same
partition was emitted as both a lake-only split and a Fluss-log split.
## Brief change log
- Add `PaimonPartitionUtils#partitionValues`, which renders a Paimon
partition
`BinaryRow` into Fluss partition-name strings in a logical-type-aware way,
matching `PartitionUtils#convertValueOfType` (the Fluss-side format).
- Compute the partition values in `PaimonSplitPlanner` (where the partition
type
is available) and carry them on `PaimonSplit`; `PaimonSplit#partition()`
now
returns them instead of re-deriving via `getString`.
- Apply the same fix to
`DvTableReadableSnapshotRetriever#getPartitionNameFromBinaryRow`,
which had the identical `getString`-based bug on the PK/DV path.
## Tests
- `PaimonPartitionUtilsTest`: every partition type's output equals
`PartitionUtils#convertValueOfType` for the same value; plus string and
multi-column cases.
- `PaimonSplitTest#testPaimonSplitWithDatePartition`: a DATE-partitioned
table
yields `["2024-03-01"]` end-to-end through the planner (garbage before).
- `PaimonSplitSerializerTest`: partition values round-trip through
serialization.
## API and Format
`PaimonSplit`'s serialized form now includes the partition values (appended
after
the existing fields).
## Documentation
No user-facing documentation change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
