I am trying to use partitionsFor method of kafka client 2.11: 0.10.2.1 <https://mvnrepository.com/artifact/org.apache.kafka/kafka_2.11/0.10.2.1> for topics published by Kafka 0.8.2.x, but it gives timeout. Following are more details:
I have a small Java spark service which uses kafka client 2.11: 0.10.2.1 <https://mvnrepository.com/artifact/org.apache.kafka/kafka_2.11/0.10.2.1>. Following is the code which works fine when I read topics published from latest Kafka version: Properties props = new Properties(); props.put(org.apache.kafka.clients.producer.ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, producerConfig.getBrokerConnectionString()); props.put(org.apache.kafka.clients.producer.ProducerConfig.ACKS_CONFIG, "all"); props.put(org.apache.kafka.clients.producer.ProducerConfig.RETRIES_CONFIG, producerConfig.getRetry()); props.put(org.apache.kafka.clients.producer.ProducerConfig.BATCH_SIZE_CONFIG, producerConfig.getBatchSize()); props.put(org.apache.kafka.clients.producer.ProducerConfig.LINGER_MS_CONFIG, producerConfig.getLingerTimeInMs()); props.put(org.apache.kafka.clients.producer.ProducerConfig.REQUEST_TIMEOUT_MS_CONFIG, producerConfig.getRequestTimeout()); props.put(org.apache.kafka.clients.producer.ProducerConfig.MAX_BLOCK_MS_CONFIG, producerConfig.getMaxBlockMS()); props.put(org.apache.kafka.clients.producer.ProducerConfig.CONNECTIONS_MAX_IDLE_MS_CONFIG, producerConfig.getMaxIdleTime()); props.put(org.apache.kafka.clients.producer.ProducerConfig.BUFFER_MEMORY_CONFIG, maxBytesInBuffer / producerConfig.getProducersCount()); props.put(org.apache.kafka.clients.producer.ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringSerializer"); props.put(org.apache.kafka.clients.producer.ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.ByteArraySerializer"); producers = new Producer[1]; producers[0] = new KafkaProducer<>(props); producers[0].partitionsFor("mYTopic").size(); There is a existing Kafka topic, where kafka version is 0.8.2.x . I wanted to use the same code for this as well. But this code gives timeout in last line(partitionsFor) with topic published by Kafka of version 0.8.2.x. Any help in this regard will be appreciated. in short: Kafka topic(published by 0.8.2.x) not able to read by 0.10.2.1 client Best Regards Saurabh Kumar Mimani
