hddong commented on a change in pull request #1452: [HUDI-740]Fix can not
specify the sparkMaster of clean and compact commands
URL: https://github.com/apache/incubator-hudi/pull/1452#discussion_r404900670
##########
File path: hudi-cli/src/main/java/org/apache/hudi/cli/utils/SparkUtil.java
##########
@@ -61,9 +62,14 @@ public static SparkLauncher initLauncher(String
propertiesFile) throws URISyntax
}
public static JavaSparkContext initJavaSparkConf(String name) {
+ return initJavaSparkConf(name, Option.empty(), Option.empty());
+ }
+
+ public static JavaSparkContext initJavaSparkConf(String name, Option<String>
master,
+ Option<String>
executorMemory) {
SparkConf sparkConf = new SparkConf().setAppName(name);
- String defMasterFromEnv = sparkConf.getenv("SPARK_MASTER");
+ String defMasterFromEnv = master.orElse(sparkConf.getenv("SPARK_MASTER"));
Review comment:
@yanghua yes, necessary, how about add a file `HoodieCliSparkConfig`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
