lamber-ken created HUDI-663:
-------------------------------
Summary: Fix HoodieDeltaStreamer offset not handled correctly
Key: HUDI-663
URL: https://issues.apache.org/jira/browse/HUDI-663
Project: Apache Hudi (incubating)
Issue Type: Bug
Components: DeltaStreamer
Reporter: lamber-ken
Go ahead with [https://github.com/apache/incubator-hudi/issues/1375]
{code:java}
20/03/04 10:21:02 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 14.0
(TID 10502, biedva-worker-6.bigdatapoc.local, executor 7):
og.apache.kafka.clients.consumer.OffsetOutOfRangeException: Offsets out of
range with no configured reset policy for partitions: {fr-bru-0=0}
at
org.apache.kafka.clients.consumer.internals.Fetcher.parseCompletedFetch(Fetcher.java:1002)
at
org.apache.kafka.clients.consumer.internals.Fetcher.fetchedRecords(Fetcher.java:508)
at
org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1261)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1189)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1115)
at
org.apache.spark.streaming.kafka010.InternalKafkaConsumer.poll(KafkaDataConsumer.scala:200)
at
org.apache.spark.streaming.kafka010.InternalKafkaConsumer.get(KafkaDataConsumer.scala:129)
at
org.apache.spark.streaming.kafka010.KafkaDataConsumer$class.get(KafkaDataConsumer.scala:36)
at
org.apache.spark.streaming.kafka010.KafkaDataConsumer$CachedKafkaDataConsumer.get(KafkaDataConsumer.scala:212)
at
org.apache.spark.streaming.kafka010.KafkaRDDIterator.next(KafkaRDD.scala:261)
at
org.apache.spark.streaming.kafka010.KafkaRDDIterator.next(KafkaRDD.scala:229)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:410)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
