Hi, The Kafka connector currently uses the older Kafka Scala consumer. Kafka 0.9 came out with a new Java Kafka consumer.
One of the main differences is that the Scala consumer uses a Decoder( kafka.serializer.decoder) trait to decode keys/values while the Java consumer uses the Deserializer interface (org.apache.kafka.common.serialization.deserializer). The main difference between Decoder and Deserializer is that Deserializer.deserialize accepts a topic and a payload while Decoder.decode accepts only a payload. Topics in Kafka are pretty useful, as one example: Confluent Schema Registry uses topic names to find the schema for each key/value - while Confluent does provide a Decoder implementation, it is mostly a hack that is incompatible with the new Kafka Java Producer. Any thoughts about changing the Kafka connector to work with the new Kafka Java Consumer? Cheers, Eugene
