garyli1019 commented on a change in pull request #1486: [HUDI-759] Integrate
checkpoint provider with delta streamer
URL: https://github.com/apache/incubator-hudi/pull/1486#discussion_r407793452
##########
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/checkpointing/InitialCheckPointProvider.java
##########
@@ -18,14 +18,38 @@
package org.apache.hudi.utilities.checkpointing;
+import org.apache.hudi.common.config.TypedProperties;
import org.apache.hudi.exception.HoodieException;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+
/**
* Provide the initial checkpoint for delta streamer.
*/
-public interface InitialCheckPointProvider {
+public abstract class InitialCheckPointProvider {
+ protected transient Path path;
+ protected transient FileSystem fs;
+ protected transient TypedProperties props;
+
+ static class Config {
+ private static String CHECKPOINT_PROVIDER_PATH_PROP =
"hoodie.deltastreamer.checkpoint.provider.path";
+ }
+
+ public InitialCheckPointProvider(TypedProperties props) {
+ this.props = props;
+ this.path = new
Path(props.getString(Config.CHECKPOINT_PROVIDER_PATH_PROP));
+ }
+
+ /**
+ * Initialize the class with the current filesystem.
+ *
+ * @param fileSystem
+ */
+ public abstract void init(FileSystem fileSystem);
Review comment:
Good point. now it's not `hiveConf` any more :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
