nsivabalan commented on a change in pull request #3590:
URL: https://github.com/apache/hudi/pull/3590#discussion_r716254203
##########
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/HoodieSparkTable.java
##########
@@ -66,4 +77,34 @@ protected HoodieSparkTable(HoodieWriteConfig config,
HoodieEngineContext context
protected HoodieIndex<T, JavaRDD<HoodieRecord<T>>, JavaRDD<HoodieKey>,
JavaRDD<WriteStatus>> getIndex(HoodieWriteConfig config, HoodieEngineContext
context) {
return SparkHoodieIndex.createIndex(config);
}
+
+ /**
+ * Fetch instance of {@link HoodieTableMetadataWriter}.
+ *
+ * @return instance of {@link HoodieTableMetadataWriter}
+ */
+ @Override
+ public Option<HoodieTableMetadataWriter> getMetadataWriter() {
+ synchronized (this) {
+ if (!isMetadataInfoUpdated.getAndSet(true)) {
+ // this code assumes that bootstrap of metadata table is done
elsewhere (SparkRDDWriteClient) and so, we gauge whether metadata table
+ // is available or not once here and reuse the same info for repeated
calls to getMetadataWriter(). Please remove memoization if
+ // that's not the case.
+ try {
+ if (!config.isMetadataTableEnabled() ||
!metaClient.getFs().exists(new
Path(HoodieTableMetadata.getMetadataTableBasePath(metaClient.getBasePath())))) {
+ isMetadataTableAvailable = false;
+ } else {
+ isMetadataTableAvailable = true;
+ }
+ } catch (IOException e) {
+ throw new HoodieMetadataException("Could not instantiate metadata
table writer", e);
+ }
+ }
+ }
+ if (isMetadataTableAvailable) {
Review comment:
isMetadataTableAvailable is not an Optional element. it is just a
boolean. and so I could not do Option.of(boolean).filter()... I will keep
trying out few things here if I can get rid of if else. for now, will leave it
as is. I could make isMetadataTableAvailable an Option<boolean> and set it
explicitly within the sync block only if its avaialble and leave it as
Option.empty if metadata is not available, but feel its not neat either.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
