SEPURI-SAI-KRISHNA opened a new pull request, #19710:
URL: https://github.com/apache/hudi/pull/19710
### Describe the issue this Pull Request addresses
Closes #19668.
`HoodieTableConfigUtils#inferPartitionValueExtractorClass` picks
`SlashEncodedDayPartitionValueExtractor` for a table written with
`hoodie.datasource.write.slash.separated.date.partitioning=true`. That
extractor requires the
partition path to be exactly three segments and throws otherwise:
```java
String[] splits = partitionPath.split("/");
if (splits.length != 3) {
throw new IllegalArgumentException("Partition path " + partitionPath + "
is not in the form yyyy/mm/dd ");
}
```
A null or empty partition value is written to the single-segment
`__HIVE_DEFAULT_PARTITION__`
directory, not to a `yyyy/MM/dd` one, so hive sync of a slash table that has
ever seen a null
partition value fails with:
```
java.lang.IllegalArgumentException: Partition path
__HIVE_DEFAULT_PARTITION__ is not in the form yyyy/mm/dd
```
A null partition value is ordinary — it is what every write path produces
for a null or empty
date column, and #19648 adds coverage for exactly that directory.
### Summary and Changelog
Both slash-encoded extractors now recognise the default-partition directory
and pass it through
instead of trying to parse it as a date. `__HIVE_DEFAULT_PARTITION__` is
Hive's own representation
of a null partition value, so handing it back unchanged is what lets Hive
map the partition to
NULL.
* `SlashEncodedDayPartitionValueExtractor#extractPartitionValuesInPath`
returns
`Collections.singletonList(DEFAULT_PARTITION_PATH)` for the
default-partition directory, before
the three-segment check.
* `SlashEncodedHourPartitionValueExtractor` gets the same treatment. It is
not auto-inferred, but
it carries the identical defect against its own four-segment check, is
user-configurable, and is
equally exposed to a null partition value — fixing one and leaving the
other seemed worse than
fixing both.
* The check tolerates a Hive-style prefix
(`datestr=__HIVE_DEFAULT_PARTITION__`), matching the way
the existing parsing already strips a `key=` prefix from each segment.
Tests in `TestPartitionValueExtractor`: the default-partition directory,
bare and Hive-style
prefixed, for both extractors. `testDayPartition` is new — the day extractor
had no test at all
despite being the auto-inferred one — and also pins the ordinary
`yyyy/MM/dd` and
`datestr=yyyy/MM/dd` cases plus the existing rejection of a malformed path.
Both new cases fail on the unfixed code with the `IllegalArgumentException`
quoted above.
### Impact
Hive sync of a slash-partitioned table containing a null partition value
succeeds instead of
failing outright. No change for any other partition path: the new branch
matches only the
default-partition marker, and everything else takes the pre-existing parsing.
The deprecated `default` marker is deliberately not handled here; the
current write paths produce
`__HIVE_DEFAULT_PARTITION__`, and mapping legacy directories would be a
separate behavior decision.
### Risk Level
low
One early-return in each of two extractors, matching a single constant, with
the pre-existing
parsing untouched. Verified with `TestPartitionValueExtractor` (4 tests, all
passing) and by
re-running the new cases against the unfixed code to confirm they fail there.
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
