sbourkeostk opened a new pull request #9482: URL: https://github.com/apache/pulsar/pull/9482
Fixes #9480 ### Motivation The kafka source sets auto.offset.reset to "earliest". This means all old messages from kafka are produced to pulsar. Often is it desirable to start form the present location "latest". The option is set after the user config has been loaded so it cannot be changed: [source code link](https://github.com/apache/pulsar/blob/master/pulsar-io/kafka/src/main/java/org/apache/pulsar/io/kafka/KafkaAbstractSource.java#L87) ### Modifications Added an autoOffsetReset option to KafkaSourceConfig ### Verifying this change - [x] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change added tests and can be verified as follows: Added to unit test and verified behaviour. ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): no - The public API: no - The schema: no - The default values of configurations: no - The wire protocol: no - The rest endpoints: no - The admin cli options: no - Anything that affects deployment: no ### Documentation - Does this pull request introduce a new feature? yes - If yes, how is the feature documented? docs / JavaDocs - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
