Hi Pratyaksh, For boolean flags, you don't need to pass true or false. It is implicit. Just pass "--enable-hive-sync" without additional true/false in the command line. Balaji.V On Friday, September 13, 2019, 03:06:38 AM PDT, Pratyaksh Sharma <pratyaks...@gmail.com> wrote: Hi,
I am trying to run HoodieDeltaStreamer and am working on tag hoodie-0.4.7. I am using spark version 2.3.2. I was trying to enable hive sync along with running HoodieDeltaStreamer by passing the flag --enable-hive-sync as true. Here is the command I used - spark-submit --master local[1] --class com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer /home/ubuntu/pratyaksh/hoodie-utilities-debezium-0.4.7.jar --enable-hive-sync false --storage-type COPY_ON_WRITE --source-class com.uber.hoodie.utilities.sources.JsonDFSSource --source-ordering-field xxxx --target-base-path hdfs://path/to/cow_table --target-table cow_table --props hdfs://path/to/fg-kafka-source.properties --schemaprovider-class com.uber.hoodie.utilities.schema.FilebasedSchemaProvider However I got the below exception - Exception in thread "main" com.uber.hoodie.com.beust.jcommander.ParameterException: Was passed main parameter 'false' but no main parameter was defined at com.uber.hoodie.com.beust.jcommander.JCommander.getMainParameter(JCommander.java:914) at com.uber.hoodie.com.beust.jcommander.JCommander.parseValues(JCommander.java:759) at com.uber.hoodie.com.beust.jcommander.JCommander.parse(JCommander.java:282) at com.uber.hoodie.com.beust.jcommander.JCommander.parse(JCommander.java:265) at com.uber.hoodie.com.beust.jcommander.JCommander.<init>(JCommander.java:210) at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer.main(HoodieDeltaStreamer.java:493) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:904) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) 19/09/12 13:59:16 INFO ShutdownHookManager: Shutdown hook called I was able to fix this by making the variable enableHiveSync in HoodieDeltaStreamer.Config class as a integer type. Has anybody faced the above issue? Any leads will be appreciated.
