The GetKafka processor has a couple of Kafka Consumer Config values that
are hard-coded.
props.setProperty("auto.commit.enable", "true"); // just be explicit
props.setProperty("auto.offset.reset", "smallest");
These should be configurable property values in the Processor. Most
notable for me is the "auto.offset.reset". Smallest vs. Largest has some
implications concerning fault tolerance strategies.
It would be best to expose all of the available Kafka Consumer Config
properties. If these change though between kafka versions it would create
maintenance work for the Processors.
Another option would be to allow ad-hoc property values and end-user just
supply the kafka config values they want to override.
*Jeremiah Adams*
Senior Software Developer
Pearson
2154 East Commons Ave.
Suite 400
Centennial, CO 80122
Always Learning
Learn more at www.pearson.com
