soumilshah1995 commented on issue #351:
URL: 
https://github.com/apache/incubator-xtable/issues/351#issuecomment-1974905580

   @the-other-tim-brown  Still getting Error 
   ```
   spark-submit \
       --class org.apache.hudi.utilities.streamer.HoodieStreamer \
       --packages 'org.apache.hudi:hudi-spark3.4-bundle_2.12:0.14.0' \
       --properties-file spark-config.properties \
       --master 'local[*]' \
       --executor-memory 1g \
       --jars 
'/Users/soumilshah/IdeaProjects/SparkProject/DeltaStreamer/jar/hudi-extensions-0.1.0-SNAPSHOT-bundled.jar'
 \
        
/Users/soumilshah/IdeaProjects/SparkProject/DeltaStreamer/jar/hudi-utilities-slim-bundle_2.12-0.14.0.jar
 \
       --table-type COPY_ON_WRITE \
       --target-base-path 
'file:///Users/soumilshah/IdeaProjects/SparkProject/DeltaStreamer/hudi/bronze_orders'
  \
       --target-table bronze_orders \
       --op UPSERT \
       --enable-sync \
       --sync-tool-classes 'io.onetable.hudi.sync.OneTableSyncTool' \
       --source-limit 4000000 \
       --source-ordering-field ts \
       --source-class org.apache.hudi.utilities.sources.CsvDFSSource \
       --hoodie-conf 'hoodie.datasource.write.recordkey.field=order_id' \
       --hoodie-conf 'hoodie.datasource.write.partitionpath.field=order_date' \
       --hoodie-conf 'hoodie.datasource.write.precombine.field=ts' \
       --hoodie-conf 
'hoodie.streamer.source.dfs.root=file://///Users/soumilshah/IdeaProjects/SparkProject/DeltaStreamer/sampledata/orders'
 \
       --hoodie-conf 'hoodie.deltastreamer.csv.header=true' \
       --hoodie-conf 'hoodie.deltastreamer.csv.sep=\t' \
       --hoodie-conf 'hoodie.onetable.formats.to.sync=DELTA'
   
   ```
   
   
   # Error 
   ```
   4/03/02 15:47:34 INFO FileSystemViewManager: Creating in-memory based Table 
View
   24/03/02 15:47:34 WARN StreamSync: SyncTool class 
io.onetable.hudi.sync.OneTableSyncTool failed with exception
   org.apache.hudi.exception.HoodieMetaSyncException: Could not sync using the 
meta sync class io.onetable.hudi.sync.OneTableSyncTool
        at 
org.apache.hudi.sync.common.util.SyncUtilHelpers.runHoodieMetaSync(SyncUtilHelpers.java:81)
        at 
org.apache.hudi.utilities.streamer.StreamSync.runMetaSync(StreamSync.java:938)
        at 
org.apache.hudi.utilities.streamer.StreamSync.writeToSink(StreamSync.java:851)
        at 
org.apache.hudi.utilities.streamer.StreamSync.syncOnce(StreamSync.java:446)
        at 
org.apache.hudi.utilities.streamer.HoodieStreamer$StreamSyncService.ingestOnce(HoodieStreamer.java:840)
        at 
org.apache.hudi.utilities.ingestion.HoodieIngestionService.startIngestion(HoodieIngestionService.java:72)
        at org.apache.hudi.common.util.Option.ifPresent(Option.java:97)
        at 
org.apache.hudi.utilities.streamer.HoodieStreamer.sync(HoodieStreamer.java:205)
        at 
org.apache.hudi.utilities.streamer.HoodieStreamer.main(HoodieStreamer.java:584)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at 
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at 
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1020)
        at 
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:192)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:215)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91)
        at 
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1111)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1120)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
   Caused by: java.util.ServiceConfigurationError: 
io.onetable.spi.sync.TargetClient: io.onetable.hudi.HudiTargetClient Unable to 
get public no-arg constructor
        at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:582)
        at 
java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:673)
        at 
java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1233)
        at 
java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1265)
        at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1300)
        at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1385)
        at 
io.onetable.client.TableFormatClientFactory.createTargetClientForName(TableFormatClientFactory.java:67)
        at 
io.onetable.client.TableFormatClientFactory.createForFormat(TableFormatClientFactory.java:51)
        at 
io.onetable.client.OneTableClient.lambda$sync$0(OneTableClient.java:99)
        at 
java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:178)
        at 
java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
        at 
java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
        at 
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at 
java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at 
java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
        at 
java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at 
java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
        at io.onetable.client.OneTableClient.sync(OneTableClient.java:95)
        at 
io.onetable.hudi.sync.OneTableSyncTool.syncHoodieTable(OneTableSyncTool.java:80)
        at 
org.apache.hudi.sync.common.util.SyncUtilHelpers.runHoodieMetaSync(SyncUtilHelpers.java:79)
        ... 20 more
   Caused by: java.lang.NoClassDefFoundError: 
org/apache/hudi/client/common/HoodieJavaEngineContext
        at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
        at 
java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3137)
        at java.base/java.lang.Class.getConstructor0(Class.java:3342)
        at java.base/java.lang.Class.getConstructor(Class.java:2151)
        at java.base/java.util.ServiceLoader$1.run(ServiceLoader.java:660)
        at java.base/java.util.ServiceLoader$1.run(ServiceLoader.java:657)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at 
java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:668)
        ... 38 more
   Caused by: java.lang.ClassNotFoundException: 
org.apache.hudi.client.common.HoodieJavaEngineContext
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
        ... 46 more
   24/03/02 15:47:34 INFO StreamSync: Shutting down embedded timeline server
   24/03/02 15:47:34 INFO EmbeddedTimelineService: Closing Timeline server
   24/03/02 15:47:34 INFO TimelineService: Closing Timeline Service
   24/03/02 15:47:34 INFO Javalin: Stopping Javalin ...
   24/03/02 15:47:34 INFO Javalin: Javalin has stopped
   24/03/02 15:47:34 INFO TimelineService: Closed Timeline Service
   24/03/02 15:47:34 INFO EmbeddedTimelineService: Closed Timeline server
   24/03/02 15:47:34 INFO SparkContext: SparkContext is stopping with exitCode 
0.
   24/03/02 15:47:34 INFO SparkUI: Stopped Spark web UI at 
http://soumils-mbp:8090
   24/03/02 15:47:34 INFO MapOutputTrackerMasterEndpoint: 
MapOutputTrackerMasterEndpoint stopped!
   24/03/02 15:47:34 INFO MemoryStore: MemoryStore cleared
   24/03/02 15:47:34 INFO BlockManager: BlockManager stopped
   24/03/02 15:47:34 INFO BlockManagerMaster: BlockManagerMaster stopped
   24/03/02 15:47:34 INFO 
OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: 
OutputCommitCoordinator stopped!
   24/03/02 15:47:34 INFO SparkContext: Successfully stopped SparkContext
   Exception in thread "main" 
org.apache.hudi.exception.HoodieMetaSyncException: Could not sync using the 
meta sync class io.onetable.hudi.sync.OneTableSyncTool
        at 
org.apache.hudi.sync.common.util.SyncUtilHelpers.runHoodieMetaSync(SyncUtilHelpers.java:81)
        at 
org.apache.hudi.utilities.streamer.StreamSync.runMetaSync(StreamSync.java:938)
        at 
org.apache.hudi.utilities.streamer.StreamSync.writeToSink(StreamSync.java:851)
        at 
org.apache.hudi.utilities.streamer.StreamSync.syncOnce(StreamSync.java:446)
        at 
org.apache.hudi.utilities.streamer.HoodieStreamer$StreamSyncService.ingestOnce(HoodieStreamer.java:840)
        at 
org.apache.hudi.utilities.ingestion.HoodieIngestionService.startIngestion(HoodieIngestionService.java:72)
        at org.apache.hudi.common.util.Option.ifPresent(Option.java:97)
        at 
org.apache.hudi.utilities.streamer.HoodieStreamer.sync(HoodieStreamer.java:205)
        at 
org.apache.hudi.utilities.streamer.HoodieStreamer.main(HoodieStreamer.java:584)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at 
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at 
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1020)
        at 
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:192)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:215)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91)
        at 
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1111)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1120)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
   Caused by: java.util.ServiceConfigurationError: 
io.onetable.spi.sync.TargetClient: io.onetable.hudi.HudiTargetClient Unable to 
get public no-arg constructor
        at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:582)
        at 
java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:673)
        at 
java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1233)
        at 
java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1265)
        at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1300)
        at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1385)
        at 
io.onetable.client.TableFormatClientFactory.createTargetClientForName(TableFormatClientFactory.java:67)
        at 
io.onetable.client.TableFormatClientFactory.createForFormat(TableFormatClientFactory.java:51)
        at 
io.onetable.client.OneTableClient.lambda$sync$0(OneTableClient.java:99)
        at 
java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:178)
        at 
java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
        at 
java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
        at 
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at 
java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at 
java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
        at 
java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at 
java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
        at io.onetable.client.OneTableClient.sync(OneTableClient.java:95)
        at 
io.onetable.hudi.sync.OneTableSyncTool.syncHoodieTable(OneTableSyncTool.java:80)
        at 
org.apache.hudi.sync.common.util.SyncUtilHelpers.runHoodieMetaSync(SyncUtilHelpers.java:79)
        ... 20 more
   Caused by: java.lang.NoClassDefFoundError: 
org/apache/hudi/client/common/HoodieJavaEngineContext
        at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
        at 
java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3137)
        at java.base/java.lang.Class.getConstructor0(Class.java:3342)
        at java.base/java.lang.Class.getConstructor(Class.java:2151)
        at java.base/java.util.ServiceLoader$1.run(ServiceLoader.java:660)
        at java.base/java.util.ServiceLoader$1.run(ServiceLoader.java:657)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at 
java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:668)
        ... 38 more
   Caused by: java.lang.ClassNotFoundException: 
org.apache.hudi.client.common.HoodieJavaEngineContext
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
        ... 46 more
   24/03/02 15:47:34 INFO ShutdownHookManager: Shutdown hook called
   24/03/02 15:47:34 INFO ShutdownHookManager: Deleting directory 
/private/var/folders/qq/s_1bjv516pn_mck29cwdwxnm0000gp/T/spark-4590b9e3-daae-4317-9782-c7126533de25
   24/03/02 15:47:34 INFO ShutdownHookManager: Deleting directory 
/private/var/folders/qq/s_1bjv516pn_mck29cwdwxnm0000gp/T/spark-ea3fcde5-0a6c-46e7-aad7-264fd4c4c8f6
   soumilshah@Soumils-MBP DeltaStreamer % 
   
   ```
   
   
   would it be best for quick call to resolve it ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to