yihua commented on code in PR #19304:
URL: https://github.com/apache/hudi/pull/19304#discussion_r4102161241
##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java:
##########
@@ -1595,6 +1605,16 @@ public Properties build() {
KeyGeneratorType type = KeyGeneratorType.valueOf(keyGeneratorType);
tableConfig.setValue(HoodieTableConfig.KEY_GENERATOR_TYPE,
type.name());
}
+ if (tableConfig.isComplexKeyGenWithSingleRecordKeyField() &&
metaFieldsMode.isRecordKeyPopulated()) {
+ // New tables store `<field>:<value>` keys. Only releases up to 1.0.2
(table version 8 and below) ever
+ // wrote bare values, so that encoding can only be declared for a
table created at such a version.
+ ComplexKeyGenEncoding encoding = complexKeyGenEncoding != null ?
complexKeyGenEncoding : ComplexKeyGenEncoding.FIELD_PREFIXED;
+ checkArgument(encoding == ComplexKeyGenEncoding.FIELD_PREFIXED ||
tableVersion.lesserThan(HoodieTableVersion.NINE),
+ String.format("%s=%s is only valid for a table created at version
%s or below; tables at version %s and above "
Review Comment:
Yes -- a pre-v9 VALUE_ONLY table upgrades to v9/v10 and keeps VALUE_ONLY.
`TestComplexKeyGenExistingTableUpgrade#testUpgradeToCurrentVersionWithDefaultsNoDuplicates`
asserts exactly that, and the property is in
`CONFIGS_REQUIRED_FOR_OLDER_VERSIONED_TABLES` so it survives version filtering.
The v9 restriction only applies at table *creation*, and that part is by
design: a newly created v9+ table can only be FIELD_PREFIXED, since no writer
at that version produces bare keys. So the CI failure is a test bug rather than
a guard problem --
`TestHoodieTableSource#testRecordLevelIndexPruningHonorsRecordedEncoding[2]`
creates its table at the current version with VALUE_ONLY, whereas
`TestComplexKeyGenEncodingWrite#testWriteToLegacyTableWithoutRecordedEncoding`
sets `WRITE_TABLE_VERSION = EIGHT` first and passes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
