[
https://issues.apache.org/jira/browse/HUDI-3450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-3450:
---------------------------------
Labels: pull-request-available (was: )
> Avoid passing empty string spark master to hudi cli
> ---------------------------------------------------
>
> Key: HUDI-3450
> URL: https://issues.apache.org/jira/browse/HUDI-3450
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Wenning Ding
> Priority: Major
> Labels: pull-request-available
>
> When using Hudi CLI, when not passing SparkMaster, by default Hudi CLI should
> use
> [SparkUtil.DEFAULT_SPARK_MASTER|https://github.com/apache/hudi/blob/release-0.10.0/hudi-cli/src/main/java/org/apache/hudi/cli/utils/SparkUtil.java#L44].
> However, w/ a recent [code
> change|https://github.com/apache/hudi/commit/445208a0d20b457daeeb5f70995302c92dd19f31]
> in OSS, when SparkMaster is not passed, it would set Spark master to {{""}}
> which causes the following exception when initializing a Hudi CLI job:
>
> {{org.apache.spark.SparkException: Could not parse Master URL: ''at
> org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2999)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:567)
> at
> org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58)
> at org.apache.hudi.cli.utils.SparkUtil.initJavaSparkConf(SparkUtil.java:115)
> at org.apache.hudi.cli.utils.SparkUtil.initJavaSparkConf(SparkUtil.java:110)
> at org.apache.hudi.cli.commands.SparkMain.main(SparkMain.java:88)}}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
