Hi, I have 3 node kafka cluster. I have one kafka consumer which will listen to particular topic. Facing following issue:
08:52:25.733 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name heartbeat-latency 08:52:25.733 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name join-latency 08:52:25.734 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name sync-latency 08:52:25.739 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name commit-latency 08:52:25.746 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-fetched 08:52:25.747 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-fetched 08:52:25.747 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-latency 08:52:25.747 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-lag 08:52:25.747 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-throttle-time 08:52:25.748 [SparkApp-akka.actor.default-dispatcher-2] WARN o.a.k.c.consumer.ConsumerConfig - The configuration autooffset.reset = earliest was supplied but isn't a known config. 08:52:25.748 [SparkApp-akka.actor.default-dispatcher-2] WARN o.a.k.c.consumer.ConsumerConfig - The configuration auto.commit = true was supplied but isn't a known config. 08:52:25.749 [SparkApp-akka.actor.default-dispatcher-2] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version : 0.10.0.1 08:52:25.749 [SparkApp-akka.actor.default-dispatcher-2] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId : a7a17cdec9eaa6c5 08:52:25.750 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.k.clients.consumer.KafkaConsumer - Kafka consumer created 08:52:25.750 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.k.clients.consumer.KafkaConsumer - Subscribed to partition(s): orchestrator-0 08:52:25.751 [SparkApp-akka.actor.default-dispatcher-2] DEBUG o.a.k.clients.consumer.KafkaConsumer - Seeking to offset 0 for partition orchestrator-0 08:52:26.756 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending coordinator request for group NewCustomers to broker datapipeline-us-west-2-kafka-02.sse.com:9092 (id: -2 rack: null) 08:52:26.773 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -2 at datapipeline-us-west-2-kafka-02.sse.com:9092. 08:52:26.827 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.apache.kafka.clients.NetworkClient - Initialize connection to node -3 for sending metadata request 08:52:26.827 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -3 at datapipeline-us-west-2-kafka-03.sse.com:9092. 08:52:26.929 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.apache.kafka.clients.NetworkClient - Initialize connection to node -1 for sending metadata request 08:52:26.929 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -1 at datapipeline-us-west-2-kafka-01.sse.com:9092. 08:52:30.739 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown 08:52:30.838 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown 08:52:30.938 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown 08:52:31.039 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown 08:52:31.139 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown 08:52:31.239 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown 08:52:31.340 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown 08:52:31.439 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown 08:52:31.539 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown 08:52:31.640 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown 08:52:31.740 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown 08:52:31.840 [SparkApp-akka.actor.default-dispatcher-9] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Cannot auto-commit offsets for group NewCustomers since the coordinator is unknown Please let me know if anyone come across above issue Regards, Ashwini
