[
https://issues.apache.org/jira/browse/HUDI-9093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen closed HUDI-9093.
----------------------------
Reviewers: Danny Chen
Resolution: Fixed
Fixed via master branch: c002cdf340622a345045cf4d59c8dd2518152158
> HoodieClusteringJob using duplicate alias "-sc" for two different parameters
> ----------------------------------------------------------------------------
>
> Key: HUDI-9093
> URL: https://issues.apache.org/jira/browse/HUDI-9093
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Mansi Patel
> Assignee: Mansi Patel
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> HoodieClusteringJob {color:#1f2328}has bean set two options with the same
> "-sc" parameter.{color}
> [https://github.com/apache/hudi/blob/master/hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieClusteringJob.java#L95-L98]
> *{color:#1f2328}Source Code:{color}*
> {code:java}
> @Parameter(names = {"--skip-clean", "-sc"}, description = "do not trigger
> clean after clustering", required = false)
> public Boolean skipClean = true;
> @Parameter(names = {"--schedule", "-sc"}, description = "Schedule clustering
> @desperate soon please use \"--mode schedule\" instead")
> public Boolean runSchedule = false;
> {code}
> *{color:#1f2328}Spark-Submit command:{color}*
> {code:java}
> spark-submit \
> --class org.apache.hudi.utilities.HoodieClusteringJob \
>
> packaging/hudi-utilities-bundle/target/hudi-utilities-bundle_2.12-1.1.0-SNAPSHOT.jar
> \
> --base-path <PATH> \
> --table-name <TABLE_NAME> \
> --spark-master local \
> --instant-time 20250227233500970 \
> --mode scheduleAndExecute {code}
> {color:#1f2328}*Exception:*{color}
> {code:java}
> 25/02/28 23:39:50 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to
> another address
> Exception in thread "main"
> org.apache.hudi.com.beust.jcommander.ParameterException: Found the option -sc
> multiple times
> at
> org.apache.hudi.com.beust.jcommander.JCommander.addDescription(JCommander.java:627)
> at
> org.apache.hudi.com.beust.jcommander.JCommander.createDescriptions(JCommander.java:594)
> at
> org.apache.hudi.com.beust.jcommander.JCommander.<init>(JCommander.java:249)
> at
> org.apache.hudi.utilities.HoodieClusteringJob.main(HoodieClusteringJob.java:149)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> at
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1034)
> at
> org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:199)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:222)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91)
> at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1125)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1134)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> {code}
> {color:#1f2328}*Issue Reference:*{color}
> {color:#1f2328}[https://github.com/apache/hudi/issues/12832]{color}
> {color:#1f2328}Similar issue for HoodieCompactor:
> https://issues.apache.org/jira/browse/HUDI-9016{color}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
