yihua opened a new pull request, #19566:
URL: https://github.com/apache/hudi/pull/19566
### Describe the issue this Pull Request addresses
Closes #19562
### Summary and Changelog
`hoodie.timeline.path` and `hoodie.timeline.history.path` describe the
timeline layout version 2 folders, introduced with table version 8. A table
below that version keeps its timeline directly under `.hoodie` and resolves its
archived timeline through `hoodie.archivelog.folder`, so neither config has
meaning there.
Neither declared a `sinceVersion`, and
`HoodieTableConfig.dropInvalidConfigs()` only strips a config that declares
one, so both were persisted into the `hoodie.properties` of tables that do not
use the layout they describe. `RECORD_MERGE_MODE` declares
`.sinceVersion("1.0.0")` and is correctly dropped from a version 6 table, which
is the behaviour these two were missing.
Changes:
- Declare `sinceVersion("1.0.0")` on `TIMELINE_PATH` and
`TIMELINE_HISTORY_PATH` so the existing gate drops them below table version 8.
- Add `TestHoodieTableConfig` coverage across table versions 6, 7, 8 and 9,
with literal per-version expectations, also asserting that `ARCHIVELOG_FOLDER`
is retained at every version.
`hoodie.table.format` was considered alongside these and deliberately left
alone. Its introducing version is not the version at which it becomes
meaningful: the table format SPI is orthogonal to the table version, nothing in
the codebase gates a custom format on table version 9, and no upgrade handler
restores the config. Gating it on `sinceVersion` would silently and permanently
discard a custom format on any table below version 9, so excluding that config
needs a different mechanism.
### Impact
Tables created below version 8 no longer carry the two timeline layout
version 2 configs in their properties file.
No read path changes. Every consumer resolves these through
`getStringOrDefault`, and the only provider that reads them,
`TimelinePathProviderV2`, is selected exclusively at layout version 2 where the
configs are retained. `TimelinePathProviderV1` reads neither. Resolved timeline
and archive paths were verified unchanged at table versions 6, 7, 8 and 9.
Existing tables are not modified; the gate applies when properties are
written. Upgrade is unaffected, since `SevenToEightUpgradeHandler` re-adds
`TIMELINE_PATH` through `HoodieTableConfig.update()`, which does not run the
gate.
One cosmetic inconsistency remains: a table upgraded from 6 or 7 to 8 will
not carry `hoodie.timeline.history.path`, because that handler adds
`hoodie.timeline.path` but not its sibling. Both resolve to the same default
either way.
No public API change.
### Risk Level
low
Confined to which keys are written into `hoodie.properties`. Verified that
no consumer reads either config through a null-returning accessor, and that
resolved paths are identical before and after across four table versions.
Reverting the change fails the added test at versions 6 and 7.
### Documentation Update
none
No config is added or removed and no default value changes; this records the
version at which two existing configs were introduced.
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
