rahil-c commented on code in PR #13650:
URL: https://github.com/apache/hudi/pull/13650#discussion_r2289019036
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java:
##########
@@ -1689,4 +1694,22 @@ private InternalSchema
getInternalSchema(TableSchemaResolver schemaUtil) {
}
});
}
+
+ private void validateComplexKeygen(HoodieTableMetaClient metaClient) {
+ HoodieTableConfig tableConfig = metaClient.getTableConfig();
+ String keyGeneratorClassName = tableConfig.getKeyGeneratorClassName();
+ Option<String[]> recordKeyFields = tableConfig.getRecordKeyFields();
+ if ((SPARK_COMPLEX_KEYGEN_CLASS_NAME.equals(keyGeneratorClassName)
Review Comment:
@danny0405 How do we know if this truly complex key gen case though, and not
some other key generator? For example I think `CUSTOM` key generator
https://hudi.apache.org/docs/key_generation/#custom can also follow the same
properties you described such as using one field but multiple partition fields.
```
https://hudi.apache.org/docs/key_generation/#custom
hoodie.datasource.write.keygenerator.class=org.apache.hudi.keygen.CustomKeyGenerator
This keyGenerator is particularly useful if you want to define complex
partition paths involving regular fields and timestamp based fields. It expects
value for prop "hoodie.datasource.write.partitionpath.field" in a specific
format. The format should be
"field1:PartitionKeyType1,field2:PartitionKeyType2..."
```
@yihua I think checking the table config makes sense to me, which should be
engine agnostic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
