nsivabalan commented on code in PR #8107:
URL: https://github.com/apache/hudi/pull/8107#discussion_r1154469420
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -1145,6 +1145,10 @@ public String getKeyGeneratorClass() {
return getString(KEYGENERATOR_CLASS_NAME);
}
+ public boolean autoGenerateRecordKeys() {
+ return getBooleanOrDefault(HoodieTableConfig.AUTO_GENERATE_RECORD_KEYS);
+ }
Review Comment:
yes, I had a jam w/ vinoth yesterday. we are going to rely on user
configuring record key. If its configured we will honor it. if not, we will
auto generate it. wrt operation type, its little involved. We have to decide
the default operation type based on whether use has configured other props or
not.
my high level goal here is:
for someone using plain parquet (writing to parquet table), when they switch
to hudi, we want to give them smoother transition.
so, df.wirte.format("parquet").save(location)
should be easily translatable to
df.wirte.format("hudi").save(newLocation).
along w/ it, we can infer partitions if
df.write.partitionBy(colA).format("hudi").save(newLocation).
So, in these cases(no precombine), we should make insert as the default
operation type. In other cases, we will make upsert as default operation type
(only when user has not overridden the operation type).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
