GitHub user relief-melone added a comment to the discussion: Should Kafka Source Connector itsself after unrecoverable error?
> I think the root cause is here: > > https://github.com/apache/pulsar/blob/1c813fdaeb9f752b84347b5a62e219dd66e79a26/pulsar-io/kafka/src/main/java/org/apache/pulsar/io/kafka/KafkaAbstractSource.java#L191-L192 > > Did you see some messages like `Error while consuming records` in the > connector log? The Kafka source connector will print these exceptions but not > exit the current process. > > I think this could be considered a bug. We need to fail the connector if > there are any unexpected errors. Thanks for the PR. It seems like that could very well be the reason for the behaviour. This is the portion of the Error log in our particular case that might be interesting for you. So the *Error while consuming records* does indeed show. Happy to try it out. `2023-03-15T12:09:08,700+0000 [Kafka Source Thread] ERROR org.apache.pulsar.io.kafka.KafkaAbstractSource - [Kafka Source Thread] Error while consuming records org.apache.kafka.clients.consumer.CommitFailedException: Offset commit cannot be completed since the consumer is not part of an active group for auto partition assignment; it is likely that the consumer was kicked out of the group. at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.sendOffsetCommitRequest(ConsumerCoordinator.java:1139) ~[kafka-clients-2.7.2.jar:?] at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.commitOffsetsSync(ConsumerCoordinator.java:1004) ~[kafka-clients-2.7.2.jar:?] at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1389) ~[kafka-clients-2.7.2.jar:?] at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1343) ~[kafka-clients-2.7.2.jar:?] at org.apache.pulsar.io.kafka.KafkaAbstractSource.lambda$start$0(KafkaAbstractSource.java:174) ~[juUFGcDeAgkU9W0VTrIV-g/:?] at java.lang.Thread.run(Thread.java:829) ~[?:?]` GitHub link: https://github.com/apache/pulsar/discussions/19880#discussioncomment-6034152 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
