nsivabalan commented on a change in pull request #3315:
URL: https://github.com/apache/hudi/pull/3315#discussion_r674408672
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java
##########
@@ -608,6 +608,14 @@ public boolean populateMetaFields() {
HoodieTableConfig.HOODIE_POPULATE_META_FIELDS.defaultValue()));
}
+ public String getSimpleRecordKeyField() {
+ return getString(HoodieTableConfig.HOODIE_TABLE_SIMPLE_RECORDKEY_FIELD);
Review comment:
I do see we already have two configs in table props to hold record key
and partition path from sql layer. But I am not very sure if it works across
all layers. i.e. data source, write client etc. in other words, not sure if I
can re-use the same variables to store record keys and partition paths.
Open to discuss and decide on how to go about this.
##########
File path:
hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/hudi/DataSourceUtils.java
##########
@@ -182,6 +183,13 @@ public static HoodieWriteConfig createHoodieConfig(String
schemaStr, String base
builder = builder.withSchema(schemaStr);
}
+ boolean isSimpleKeyGen =
(parameters.getOrDefault(DataSourceWriteOptions.KEYGENERATOR_CLASS_OPT_KEY().key(),
DataSourceWriteOptions.DEFAULT_KEYGENERATOR_CLASS_OPT_VAL()))
+ .equals(SimpleKeyGenerator.class.getName());
+ if (isSimpleKeyGen) {
Review comment:
I chose to serialize the props only when key gen is of type simple. Once
we add a validation rule that meta client can be enabled only w/ simple key
gen, we can serialize these props only when meta fields are disabled. don't
really need to check for simple key gen.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
