FarhadF commented on issue #11658:
URL: https://github.com/apache/druid/issues/11658#issuecomment-1682303884
facing the same issue:
```
2023-08-17T13:35:09,349 INFO [KafkaSupervisor-staging-campaign-install]
org.apache.druid.indexing.seekablestream.supervisor.SeekableStreamSupervisor -
[staging-campaign-install] supervisor is running. │
│ 2023-08-17T13:35:24,567 INFO
[KafkaSupervisor-staging-campaign-install-Reporting-0]
org.apache.kafka.clients.consumer.internals.SubscriptionState - [Consumer
clientId=consumer-staging-campaign-install-supervisor-19, groupId=st │
│ aging-campaign-install-supervisor] Seeking to latest offset of partition
staging-campaign-install-2
│
│ 2023-08-17T13:35:24,567 INFO
[KafkaSupervisor-staging-campaign-install-Reporting-0]
org.apache.kafka.clients.consumer.internals.SubscriptionState - [Consumer
clientId=consumer-staging-campaign-install-supervisor-19, groupId=st │
│ aging-campaign-install-supervisor] Seeking to latest offset of partition
staging-campaign-install-0
│
│ 2023-08-17T13:35:24,567 INFO
[KafkaSupervisor-staging-campaign-install-Reporting-0]
org.apache.kafka.clients.consumer.internals.SubscriptionState - [Consumer
clientId=consumer-staging-campaign-install-supervisor-19, groupId=st │
│ aging-campaign-install-supervisor] Seeking to latest offset of partition
staging-campaign-install-1
│
│ 2023-08-17T13:35:24,568 INFO
[KafkaSupervisor-staging-campaign-install-Reporting-0]
org.apache.kafka.clients.consumer.internals.SubscriptionState - [Consumer
clientId=consumer-staging-campaign-install-supervisor-19, groupId=st │
│ aging-campaign-install-supervisor] Resetting offset for partition
staging-campaign-install-1 to position FetchPosition{offset=21,
offsetEpoch=Optional.empty,
currentLeader=LeaderAndEpoch{leader=Optional[staging-kafka-2.staging │
│ -kafka-brokers.kafka.svc.cluster.local:9095 (id: 2 rack: null)],
epoch=951}}.
│
│ 2023-08-17T13:35:24,568 INFO
[KafkaSupervisor-staging-campaign-install-Reporting-0]
org.apache.kafka.clients.consumer.internals.SubscriptionState - [Consumer
clientId=consumer-staging-campaign-install-supervisor-19, groupId=st │
│ aging-campaign-install-supervisor] Resetting offset for partition
staging-campaign-install-0 to position FetchPosition{offset=19,
offsetEpoch=Optional.empty,
currentLeader=LeaderAndEpoch{leader=Optional[staging-kafka-0.staging │
│ -kafka-brokers.kafka.svc.cluster.local:9095 (id: 0 rack: null)],
epoch=957}}.
│
│ 2023-08-17T13:35:24,569 INFO
[KafkaSupervisor-staging-campaign-install-Reporting-0]
org.apache.kafka.clients.consumer.internals.SubscriptionState - [Consumer
clientId=consumer-staging-campaign-install-supervisor-19, groupId=st │
│ aging-campaign-install-supervisor] Resetting offset for partition
staging-campaign-install-2 to position FetchPosition{offset=14,
offsetEpoch=Optional.empty,
currentLeader=LeaderAndEpoch{leader=Optional[staging-kafka-1.staging │
│ -kafka-brokers.kafka.svc.cluster.local:9095 (id: 1 rack: null)],
epoch=1043}}.
```
I can verify that the offsets that are reported by the logs are not ahead of
kafka offsets for each partition.
The workaround is to hard reset the supervisor but of course has the
consequence of dataloss/duplicate data
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
