rangareddy commented on issue #17003:
URL: https://github.com/apache/hudi/issues/17003#issuecomment-5391442549
This issue was reviewed as part of the JIRA-migrated backlog triage
(HUDI-9396).
**Findings: confirmed by reading the code.**
`hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/FilePathUtils.java:155`
derives the partition spec from the file's parent **path**:
```java
String[] partitionKeys = partPathField == null ||
FlinkOptions.PARTITION_PATH_FIELD.defaultValue().equals(partPathField)
? new String[0]
: partPathField.split(",");
LinkedHashMap<String, String> partSpec = extractPartitionKeyValues(
new org.apache.hadoop.fs.Path(filePath).getParent(),
hiveStylePartitioning,
partitionKeys);
```
With one partition field (`partition_path`) whose **value** contains `/` -
your `2025/10/24` case - the parent path has three directory segments but there
is only one partition key, so segment-wise extraction returns `<partition_path,
24>`. That is exactly the result you reported.
The underlying problem is that once written to the path, a single partition
value spanning multiple segments is indistinguishable from multiple partition
keys. So a fix needs either the key count and the segment count reconciled
(consume the trailing N segments for the last key), or the value recovered from
the data file rather than the path.
**The blast radius is wider than the one method** - it is called from four
Flink read paths, all of which inherit the behaviour:
- `HoodieCdcSplitReaderFunction.java:289`
- `RecordIterators.java:213`
- `table/format/cow/CopyOnWriteInputFormat.java:132`
- `table/format/mor/MergeOnReadInputFormat.java:278`
Worth noting the method already has a "fallback to file read" branch for
fields missing from the schema and a special case for datetime types, so the
precedent for reading the value from the data file instead of the path exists
in the same method.
Keeping this open.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
