Machos65 commented on issue #13184:
URL: https://github.com/apache/hudi/issues/13184#issuecomment-2828573333
so i switched to HUDI 0.15 and the error disappeared thanks for that step
but i get other error as the job run "FAILED TO CONSTRUCT KAFKA CONSUMER "
Any clue on that
Thanks
25/04/24 21:46:05 INFO SparkContext: Successfully stopped SparkContext
Exception in thread "main" org.apache.kafka.common.KafkaException: Failed to
construct kafka consumer
at
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:823)
at
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:665)
at
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:613)
at
org.apache.hudi.utilities.sources.helpers.KafkaOffsetGen.getNextOffsetRanges(KafkaOffsetGen.java:276)
at
org.apache.hudi.utilities.sources.helpers.KafkaOffsetGen.getNextOffsetRanges(KafkaOffsetGen.java:269)
at
org.apache.hudi.utilities.sources.KafkaSource.getOffsetRanges(KafkaSource.java:95)
at
org.apache.hudi.utilities.sources.KafkaSource.fetchNewData(KafkaSource.java:68)
at
org.apache.hudi.utilities.sources.AvroKafkaSource.fetchNewData(AvroKafkaSource.java:105)
at org.apache.hudi.utilities.sources.Source.fetchNext(Source.java:85)
at
org.apache.hudi.utilities.streamer.SourceFormatAdapter.fetchNewDataInAvroFormat(SourceFormatAdapter.java:158)
at
org.apache.hudi.utilities.streamer.StreamSync.fetchNextBatchFromSource(StreamSync.java:676)
at
org.apache.hudi.utilities.streamer.StreamSync.fetchFromSourceAndPrepareRecords(StreamSync.java:571)
at
org.apache.hudi.utilities.streamer.StreamSync.readFromSource(StreamSync.java:543)
at
org.apache.hudi.utilities.streamer.StreamSync.syncOnce(StreamSync.java:447)
at
org.apache.hudi.utilities.streamer.HoodieStreamer$StreamSyncService.ingestOnce(HoodieStreamer.java:876)
at
org.apache.hudi.utilities.ingestion.HoodieIngestionService.startIngestion(HoodieIngestionService.java:72)
at org.apache.hudi.common.util.Option.ifPresent(Option.java:101)
at
org.apache.hudi.utilities.streamer.HoodieStreamer.sync(HoodieStreamer.java:214)
at
org.apache.hudi.utilities.streamer.HoodieStreamer.main(HoodieStreamer.java:606)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1034)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:199)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:222)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1125)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1134)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: org.apache.kafka.common.config.ConfigException: No resolvable
bootstrap urls given in bootstrap.servers
at
org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses(ClientUtils.java:89)
at
org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses(ClientUtils.java:48)
at
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:731)
... 30 more
25/04/24 21:46:05 INFO ShutdownHookManager: Shutdown hook called
25/04/24 21:46:05 INFO ShutdownHookManager: Deleting directory
/tmp/spark-8bcf388e-0e84-4986-99bf-9698eead8d1b
25/04/24 21:46:05 INFO ShutdownHookManager: Deleting directory
/tmp/spark-6ede09ee-e329-49e1-908c-ca5c9d99a3b0
root@ICTWKS-FP002:/opt/spark#
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
