mansipp opened a new pull request, #12897:
URL: https://github.com/apache/hudi/pull/12897
### Change Logs
Fix the HoodieClusteringJob duplicate parameter alias. Corrected the alias
from "--sc" to "sch"
**Exception:**
```
25/02/28 23:39:50 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to
another address
Exception in thread "main"
org.apache.hudi.com.beust.jcommander.ParameterException: Found the option -sc
multiple times
at
org.apache.hudi.com.beust.jcommander.JCommander.addDescription(JCommander.java:627)
at
org.apache.hudi.com.beust.jcommander.JCommander.createDescriptions(JCommander.java:594)
at
org.apache.hudi.com.beust.jcommander.JCommander.<init>(JCommander.java:249)
at
org.apache.hudi.utilities.HoodieClusteringJob.main(HoodieClusteringJob.java:149)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1034)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:199)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:222)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1125)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1134)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
```
-------
**Testing the fix:**
```
spark-submit \
--class org.apache.hudi.utilities.HoodieClusteringJob \
packaging/hudi-utilities-bundle/target/hudi-utilities-bundle_2.12-1.1.0-SNAPSHOT.jar
\
--base-path <PATH> \
--table-name mansipp_hudi_mor_table \
--spark-master local \
--instant-time 20250227233500970 \
--mode scheduleAndExecute
```
```
25/03/01 00:05:32 WARN Utils: Your hostname, c889f3bb267d resolves to a
loopback address: 127.0.0.1; using 10.0.0.55 instead (on interface en0)
25/03/01 00:05:32 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to
another address
25/03/01 00:05:32 INFO SparkContext: Running Spark version 3.5.4
25/03/01 00:05:32 INFO SparkContext: OS info Mac OS X, 15.3.1, aarch64
25/03/01 00:05:32 INFO SparkContext: Java version 1.8.0_432
25/03/01 00:05:32 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
25/03/01 00:05:32 INFO ResourceUtils:
==============================================================
25/03/01 00:05:32 INFO ResourceUtils: No custom resources configured for
spark.driver.
25/03/01 00:05:32 INFO ResourceUtils:
==============================================================
25/03/01 00:05:32 INFO SparkContext: Submitted application:
clustering-mansipp_hudi_mor_table
25/03/01 00:05:32 INFO ResourceProfile: Default ResourceProfile created,
executor resources: Map(cores -> name: cores, amount: 1, script: , vendor: ,
memory -> name: memory, amount: 1024, script: , vendor: , offHeap -> name:
offHeap, amount: 0, script: , vendor: ), task resources: Map(cpus -> name:
cpus, amount: 1.0)
..................
..................
25/03/01 00:05:41 INFO MapOutputTrackerMasterEndpoint:
MapOutputTrackerMasterEndpoint stopped!
25/03/01 00:05:41 INFO MemoryStore: MemoryStore cleared
25/03/01 00:05:41 INFO BlockManager: BlockManager stopped
25/03/01 00:05:41 INFO BlockManagerMaster: BlockManagerMaster stopped
25/03/01 00:05:41 INFO
OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:
OutputCommitCoordinator stopped!
25/03/01 00:05:41 INFO SparkContext: Successfully stopped SparkContext
25/03/01 00:05:41 INFO ShutdownHookManager: Shutdown hook called
25/03/01 00:05:41 INFO ShutdownHookManager: Deleting directory
/private/var/folders/lj/hvp2tmzn6x5g4xcm8zw73v780000gq/T/spark-7255188b-ddca-4e97-bc0f-d9fc84f3a30f
25/03/01 00:05:41 INFO ShutdownHookManager: Deleting directory
/private/var/folders/lj/hvp2tmzn6x5g4xcm8zw73v780000gq/T/spark-486b4c97-e079-4a09-89b7-8d054a143d44
```
### Impact
Corrected the HoodieClusteringJob parameter alias from "--sc" to "sch"
### Risk level (write none, low medium or high below)
none
### Documentation Update
none
### Contributor's checklist
- [ ] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
