yihua commented on code in PR #13650:
URL: https://github.com/apache/hudi/pull/13650#discussion_r2308589118
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/KeyGenUtils.java:
##########
@@ -332,4 +333,21 @@ public static boolean
isAutoGeneratedRecordKeysEnabled(TypedProperties props) {
||
props.getProperty(KeyGeneratorOptions.RECORDKEY_FIELD_NAME.key()).equals(StringUtils.EMPTY_STRING);
// spark-sql sets record key config to empty string for update, and couple
of other statements.
}
+
+ public static boolean
isComplexKeyGeneratorWithSingleRecordKeyField(HoodieTableConfig tableConfig) {
+ Option<String[]> recordKeyFields = tableConfig.getRecordKeyFields();
+ return KeyGeneratorType.isComplexKeyGenerator(tableConfig)
+ && recordKeyFields.isPresent() && recordKeyFields.get().length == 1;
+ }
+
+ public static String getComplexKeygenErrorMessage(String operation) {
+ return "This table uses the complex key generator with a single record "
+ + "key field. If the table is written with Hudi 0.14.1, 0.15.0, 1.0.0,
1.0.1, or 1.0.2 "
+ + "release before, the table may potentially contain duplicates due to
a breaking "
+ + "change in the key encoding in the _hoodie_record_key meta field
(HUDI-7001) which "
+ + "is crucial for upserts. Please take action based on the mitigation
guide before "
+ + "resuming the " + operation + " to the this table. If you're certain
that the table is not "
+ + "affected by the key encoding change, set "
+ + "`hoodie.write.complex.keygen.validation.enable=false` to skip this
validation.";
Review Comment:
Actually I'll add a sentence for user to refer to the migration docs for
more details, as user might blindly set the new write config without
understanding the problem, which can cause data consistency issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
