It works. Thank you. On Fri, Sep 13, 2019 at 7:28 PM [email protected] <[email protected]> wrote:
> > Hi Pratyaksh, > For boolean flags, you don't need to pass true or false. It is implicit. > Just pass "--enable-hive-sync" without additional true/false in the command > line. > Balaji.V On Friday, September 13, 2019, 03:06:38 AM PDT, Pratyaksh > Sharma <[email protected]> wrote: > > Hi, > > I am trying to run HoodieDeltaStreamer and am working on tag hoodie-0.4.7. > I am using spark version 2.3.2. I was trying to enable hive sync along with > running HoodieDeltaStreamer by passing the flag --enable-hive-sync as true. > Here is the command I used - > > spark-submit --master local[1] --class > com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer > /home/ubuntu/pratyaksh/hoodie-utilities-debezium-0.4.7.jar > --enable-hive-sync false --storage-type COPY_ON_WRITE --source-class > com.uber.hoodie.utilities.sources.JsonDFSSource --source-ordering-field > xxxx --target-base-path hdfs://path/to/cow_table --target-table cow_table > --props hdfs://path/to/fg-kafka-source.properties --schemaprovider-class > com.uber.hoodie.utilities.schema.FilebasedSchemaProvider > > However I got the below exception - > > Exception in thread "main" > com.uber.hoodie.com.beust.jcommander.ParameterException: Was passed main > parameter 'false' but no main parameter was defined > at > > com.uber.hoodie.com.beust.jcommander.JCommander.getMainParameter(JCommander.java:914) > at > > com.uber.hoodie.com.beust.jcommander.JCommander.parseValues(JCommander.java:759) > at > com.uber.hoodie.com.beust.jcommander.JCommander.parse(JCommander.java:282) > at > com.uber.hoodie.com.beust.jcommander.JCommander.parse(JCommander.java:265) > at > com.uber.hoodie.com.beust.jcommander.JCommander.<init>(JCommander.java:210) > at > > com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer.main(HoodieDeltaStreamer.java:493) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > > org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) > at > > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:904) > at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198) > at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > 19/09/12 13:59:16 INFO ShutdownHookManager: Shutdown hook called > > I was able to fix this by making the variable enableHiveSync in > HoodieDeltaStreamer.Config class as a integer type. > > Has anybody faced the above issue? Any leads will be appreciated. >
