sc-sityad commented on issue #11658:
URL: https://github.com/apache/druid/issues/11658#issuecomment-2113111067
> facing the same issue:
>
> ```
> 2023-08-17T13:35:09,349 INFO [KafkaSupervisor-staging-campaign-install]
org.apache.druid.indexing.seekablestream.supervisor.SeekableStreamSupervisor -
[staging-campaign-install] supervisor is running. │
> │ 2023-08-17T13:35:24,567 INFO
[KafkaSupervisor-staging-campaign-install-Reporting-0]
org.apache.kafka.clients.consumer.internals.SubscriptionState - [Consumer
clientId=consumer-staging-campaign-install-supervisor-19, groupId=st │
> │ aging-campaign-install-supervisor] Seeking to latest offset of partition
staging-campaign-install-2
│
> │ 2023-08-17T13:35:24,567 INFO
[KafkaSupervisor-staging-campaign-install-Reporting-0]
org.apache.kafka.clients.consumer.internals.SubscriptionState - [Consumer
clientId=consumer-staging-campaign-install-supervisor-19, groupId=st │
> │ aging-campaign-install-supervisor] Seeking to latest offset of partition
staging-campaign-install-0
│
> │ 2023-08-17T13:35:24,567 INFO
[KafkaSupervisor-staging-campaign-install-Reporting-0]
org.apache.kafka.clients.consumer.internals.SubscriptionState - [Consumer
clientId=consumer-staging-campaign-install-supervisor-19, groupId=st │
> │ aging-campaign-install-supervisor] Seeking to latest offset of partition
staging-campaign-install-1
│
> │ 2023-08-17T13:35:24,568 INFO
[KafkaSupervisor-staging-campaign-install-Reporting-0]
org.apache.kafka.clients.consumer.internals.SubscriptionState - [Consumer
clientId=consumer-staging-campaign-install-supervisor-19, groupId=st │
> │ aging-campaign-install-supervisor] Resetting offset for partition
staging-campaign-install-1 to position FetchPosition{offset=21,
offsetEpoch=Optional.empty,
currentLeader=LeaderAndEpoch{leader=Optional[staging-kafka-2.staging │
> │ -kafka-brokers.kafka.svc.cluster.local:9095 (id: 2 rack: null)],
epoch=951}}.
│
> │ 2023-08-17T13:35:24,568 INFO
[KafkaSupervisor-staging-campaign-install-Reporting-0]
org.apache.kafka.clients.consumer.internals.SubscriptionState - [Consumer
clientId=consumer-staging-campaign-install-supervisor-19, groupId=st │
> │ aging-campaign-install-supervisor] Resetting offset for partition
staging-campaign-install-0 to position FetchPosition{offset=19,
offsetEpoch=Optional.empty,
currentLeader=LeaderAndEpoch{leader=Optional[staging-kafka-0.staging │
> │ -kafka-brokers.kafka.svc.cluster.local:9095 (id: 0 rack: null)],
epoch=957}}.
│
> │ 2023-08-17T13:35:24,569 INFO
[KafkaSupervisor-staging-campaign-install-Reporting-0]
org.apache.kafka.clients.consumer.internals.SubscriptionState - [Consumer
clientId=consumer-staging-campaign-install-supervisor-19, groupId=st │
> │ aging-campaign-install-supervisor] Resetting offset for partition
staging-campaign-install-2 to position FetchPosition{offset=14,
offsetEpoch=Optional.empty,
currentLeader=LeaderAndEpoch{leader=Optional[staging-kafka-1.staging │
> │ -kafka-brokers.kafka.svc.cluster.local:9095 (id: 1 rack: null)],
epoch=1043}}.
> ```
>
> I can verify that the offsets that are reported by the logs are not ahead
of kafka offsets for each partition. The workaround is to hard reset the
supervisor but of course has the consequence of dataloss/duplicate data
Is this resolved now? Facing the same issue, but I don't want to duplicate
the data and face any data loss by hard resetting. Please let me know if this
is fixed I don't have any task leverage to increase the task count. So any
other solution @FrankChen021
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
