Can we use the existing kafka spark streaming jar to connect to a kafka server running in SSL mode?
We are fine with non SSL consumer as our kafka cluster and spark cluster are in the same network Thanks, Sourabh On Fri, Aug 28, 2015 at 12:03 PM, Gwen Shapira <g...@confluent.io> wrote: > I can't speak for the Spark Community, but checking their code, > DirectKafkaStream and KafkaRDD use the SimpleConsumer API: > > > https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala > > https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala > > On Fri, Aug 28, 2015 at 11:32 AM, Cassa L <lcas...@gmail.com> wrote: > > > Hi I am using below Spark jars with Direct Stream API. > > spark-streaming-kafka_2.10 > > > > When I look at its pom.xml, Kafka libraries that its pulling in is > > <groupId>org.apache.kafka</groupId> > > <artifactId>kafka_${scala.binary.version}</artifactId> > > <version>0.8.2.1</version> > > > > > > I believe this DirectStream API uses SimpleConsumer API. Can someone from > > Spark community confirm too? > > > > Thanks, > > LCassa. > > > > On Fri, Aug 28, 2015 at 11:12 AM, Sriharsha Chintalapani < > ka...@harsha.io> > > wrote: > > > > > SSL is supported for new producer and consumer api and old api (simple > > > consumer and high-level consumer) is not supported. > > > I think spark uses simple consumer? if so its not supported. > > > > > > Thanks, > > > Harsha > > > > > > > > > On August 28, 2015 at 11:00:30 AM, Cassa L (lcas...@gmail.com) wrote: > > > > > > Hi, > > > I was going through SSL setup of Kafka. > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka > > > However, I am also using Spark-Kafka streaming to read data from Kafka. > > Is > > > there a way to activate SSL for spark streaming API or not possible at > > > all? > > > > > > Thanks, > > > LCassa > > > > > > > > >
