Re: Support for Kafka Consumer over SASL_SSL

2017-05-01 Thread Michal Borowiecki 

Hi Nixon,

kafka.consumer.Consumer is the old consumer.
When the documentation is referring to the new producer and consumer, it 
means those in the org.apache.kafka.clients.producer and 
org.apache.kafka.clients.consumer packages respectively.

They are packaged in the kafka-clients-0.10.x.y.jar

Hope that helps,
Michał

On 01/05/17 14:04, Nixon Rodrigues wrote:

   Hi Kafka dev team,


I am using  kafka_2.11-0.10.0.0.jar api for kafka consumer,

i am facing issue while consuming from topics over SASL_SSL,
getting below exception

Consumer.createJavaConsumerConnector(new
kafka.consumer.ConsumerConfig(consumerProperties))


As per documentation found on
http://docs.confluent.io/2.0.0/kafka/sasl.html  , it is said that

"SASL authentication is only supported for the new Kafka producer
and consumer, the older API is not supported."

Can anybody confirm me which kafka version onward SASl_SSL is
supported, any input on this is appreciated.


2017-04-26 16:06:36,655 WARN  -
[node11.openstacklocal-1493222790718-b862352a-leader-finder-thread:] ~
Fetching topic metadata with correlation id 18 for topics [Set(TEST)]
from broker [BrokerEndPoint(1001,node11.openstacklocal,6667)] failed
(Logging$class:89)
java.io.EOFException
at 
org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:99)
at kafka.network.BlockingChannel.readCompletely(BlockingChannel.scala:140)
at kafka.network.BlockingChannel.receive(BlockingChannel.scala:131)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:84)
at 
kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:81)
at kafka.producer.SyncProducer.send(SyncProducer.scala:126)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:96)
at 
kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:67)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
2017-04-26 16:06:36,655 WARN  -
[node11.openstacklocal-1493222790718-b862352a-leader-finder-thread:] ~
[node11.openstacklocal-1493222790718-b862352a-leader-finder-thread],
Failed to find leader for Set([TEST,0]) (Logging$class:89)
kafka.common.KafkaException: fetching topic metadata for topics
[Set(TEST)] from broker
[ArrayBuffer(BrokerEndPoint(1001,node11.openstacklocal,6667))] failed
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:73)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:96)
at 
kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:67)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
Caused by: java.io.EOFException
at 
org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:99)
at kafka.network.BlockingChannel.readCompletely(BlockingChannel.scala:140)
at kafka.network.BlockingChannel.receive(BlockingChannel.scala:131)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:84)
at 
kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:81)
at kafka.producer.SyncProducer.send(SyncProducer.scala:126)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)


Nixon



--
Signature
 Michal Borowiecki
Senior Software Engineer L4
T:  +44 208 742 1600


+44 203 249 8448



E:  michal.borowie...@openbet.com
W:  www.openbet.com 


OpenBet Ltd

Chiswick Park Building 9

566 Chiswick High Rd

London

W4 5XT

UK




This message is confidential and intended only for the addressee. If you 
have received this message in error, please immediately notify the 
postmas...@openbet.com  and delete it 
from your system as well as any copies. The content of e-mails as well 
as traffic data may be monitored by OpenBet for employment and security 
purposes. To protect the environment please do not print this e-mail 
unless necessary. OpenBet Ltd. Registered Office: Chiswick Park Building 
9, 566 Chiswick High Road, London, W4 5XT, United Kingdom. A company 
registered in England and Wales. Registered no. 3134634. VAT no. 
GB927523612

Support for Kafka Consumer over SASL_SSL

2017-05-01 Thread Nixon Rodrigues 
  Hi Kafka dev team,


   I am using  kafka_2.11-0.10.0.0.jar api for kafka consumer,

   i am facing issue while consuming from topics over SASL_SSL,
getting below exception

   Consumer.createJavaConsumerConnector(new
kafka.consumer.ConsumerConfig(consumerProperties))


   As per documentation found on
http://docs.confluent.io/2.0.0/kafka/sasl.html  , it is said that

   "SASL authentication is only supported for the new Kafka producer
and consumer, the older API is not supported."

   Can anybody confirm me which kafka version onward SASl_SSL is
supported, any input on this is appreciated.


2017-04-26 16:06:36,655 WARN  -
[node11.openstacklocal-1493222790718-b862352a-leader-finder-thread:] ~
Fetching topic metadata with correlation id 18 for topics [Set(TEST)]
from broker [BrokerEndPoint(1001,node11.openstacklocal,6667)] failed
(Logging$class:89)
java.io.EOFException
at 
org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:99)
at kafka.network.BlockingChannel.readCompletely(BlockingChannel.scala:140)
at kafka.network.BlockingChannel.receive(BlockingChannel.scala:131)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:84)
at 
kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:81)
at kafka.producer.SyncProducer.send(SyncProducer.scala:126)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:96)
at 
kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:67)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
2017-04-26 16:06:36,655 WARN  -
[node11.openstacklocal-1493222790718-b862352a-leader-finder-thread:] ~
[node11.openstacklocal-1493222790718-b862352a-leader-finder-thread],
Failed to find leader for Set([TEST,0]) (Logging$class:89)
kafka.common.KafkaException: fetching topic metadata for topics
[Set(TEST)] from broker
[ArrayBuffer(BrokerEndPoint(1001,node11.openstacklocal,6667))] failed
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:73)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:96)
at 
kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:67)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
Caused by: java.io.EOFException
at 
org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:99)
at kafka.network.BlockingChannel.readCompletely(BlockingChannel.scala:140)
at kafka.network.BlockingChannel.receive(BlockingChannel.scala:131)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:84)
at 
kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:81)
at kafka.producer.SyncProducer.send(SyncProducer.scala:126)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)


Nixon