danny0405 commented on code in PR #6566:
URL: https://github.com/apache/hudi/pull/6566#discussion_r961206957
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/clustering/FlinkClusteringConfig.java:
##########
@@ -116,14 +131,57 @@ public class FlinkClusteringConfig extends Configuration {
description = "Min clustering interval of async clustering service,
default 10 minutes")
public Integer minClusteringIntervalSeconds = 600;
+ @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that
can be set in the properties file "
+ + "(using the CLI parameter \"--props\") can also be passed command
line using this parameter.")
+ public List<String> configs = new ArrayList<>();
+
+ @Parameter(names = {"--props"}, description = "Path to properties file on
localfs or dfs, with configurations for "
+ + "hoodie client, schema provider, key generator and data source.
For hoodie client props, sane defaults are "
+ + "used, but recommend use to provide basic things like metrics
endpoints, hive configs etc. For sources, refer"
+ + "to individual classes, for supported properties.")
+ public String propsFilePath = "";
+
+ /**
+ * Read config from properties file (`--props` option) and cmd line
(`--hoodie-conf` option).
+ */
+ public static DFSPropertiesConfiguration
readConfig(org.apache.hadoop.conf.Configuration hadoopConfig, Path cfgPath,
List<String> overriddenProps) {
+ DFSPropertiesConfiguration conf = new
DFSPropertiesConfiguration(hadoopConfig, cfgPath);
+ try {
+ if (!overriddenProps.isEmpty()) {
+ conf.addPropsFromStream(new BufferedReader(new
StringReader(String.join("\n", overriddenProps))));
+ }
+ } catch (IOException ioe) {
+ throw new HoodieIOException("Unexpected error adding config overrides",
ioe);
Review Comment:
Can the method in `StreamerUtil.readConfig` be reused ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
