vinothchandar commented on a change in pull request #2136:
URL: https://github.com/apache/hudi/pull/2136#discussion_r528863861
##########
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/SparkRDDWriteClient.java
##########
@@ -88,7 +91,18 @@ public static SparkConf registerClasses(SparkConf conf) {
@Override
protected HoodieIndex<T, JavaRDD<HoodieRecord<T>>, JavaRDD<HoodieKey>,
JavaRDD<WriteStatus>> createIndex(HoodieWriteConfig writeConfig) {
- return SparkHoodieIndex.createIndex(config);
+ String persistIndexType = null;
+ try {
+ persistIndexType =
this.createMetaClient(true).getTableConfig().getProperties().getProperty(HoodieIndexConfig.INDEX_TYPE_PROP);
Review comment:
MetaClient does not really depend on any engine specific context. So we
could attempt that. Creating metaclient reloads the timeline again, which can
affect the `snapshot` that the writer sees. Right now, we just do it once
before we begin any write operation and its conceptually simple to understand
the rest of the code. Love to keep it that way. @lw309637554 I can take a stab
at it as well, if you feel its a bit complex. do let me know!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
