lhotari edited a comment on issue #8343: URL: https://github.com/apache/pulsar/issues/8343#issuecomment-744292914
One thing to try is to [enable "autoCommitEnabled" configuration](https://pulsar.apache.org/docs/en/io-kafka-source/#configuration) for the Kafka source connector. It should be enabled by default unless it's explicitly disabled. The reason why this might help is that Kafka Consumer API's commitSync method throws an unchecked exception according to this SO answer: https://stackoverflow.com/a/65193460 . That case might not be handled in the Pulsar IO Kafka source connector around https://github.com/apache/pulsar-connectors/blob/master/pulsar-io/kafka/src/main/java/org/apache/pulsar/io/kafka/KafkaAbstractSource.java#L139 . `commitSync` won't be called when auto commit is enabled in the Pulsar IO Kafka source connector. It might help to look at https://github.com/apache/kafka/pull/5492 to see how a similiar issue (related to handling `org.apache.kafka.common.errors.TimeoutException` in commitSync) was fixed in Kafka's Mirror Maker. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
