yanghua commented on a change in pull request #923: HUDI-247 Unify the
initialization of HoodieTableMetaClient in test for hoodie-client module
URL: https://github.com/apache/incubator-hudi/pull/923#discussion_r329094509
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java
##########
@@ -477,4 +478,43 @@ public void setActiveTimeline(HoodieActiveTimeline
activeTimeline) {
public void setTableConfig(HoodieTableConfig tableConfig) {
this.tableConfig = tableConfig;
}
+
+ //------------------------------------------------------
+ // Resource initialization and cleanup
+ //------------------------------------------------------
+
+ private void init(Configuration conf, String basePath,
+ boolean loadActiveTimelineOnLoad, ConsistencyGuardConfig
consistencyGuardConfig) {
+ log.info("Loading HoodieTableMetaClient from " + basePath);
+ this.basePath = basePath;
+ this.consistencyGuardConfig = consistencyGuardConfig;
+ this.hadoopConf = new SerializableConfiguration(conf);
+ Path basePathDir = new Path(this.basePath);
+ this.metaPath = new Path(basePath, METAFOLDER_NAME).toString();
+ Path metaPathDir = new Path(this.metaPath);
+ this.fs = getFs();
+ DatasetNotFoundException.checkValidDataset(fs, basePathDir, metaPathDir);
+ this.tableConfig = new HoodieTableConfig(fs, metaPath);
+ this.tableType = tableConfig.getTableType();
+ log.info("Finished Loading Table of type " + tableType + " from " +
basePath);
+ if (loadActiveTimelineOnLoad) {
+ log.info("Loading Active commit timeline for " + basePath);
+ getActiveTimeline();
+ }
+ }
+
+ private void cleanup() throws IOException {
+ this.basePath = null;
+ this.consistencyGuardConfig = null;
+ this.hadoopConf = null;
+ this.metaPath = null;
+ if (this.fs != null) {
Review comment:
I just see the source code of Hadoop `FileSystem`. You are right.
`FileSystem.get` will get an instance from cache, `FileSystem.newInstance` can
returns a uniquie File System object. Sorry, I did not pay attention to it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
