Do not add a dependency on kafka-clients, the spark-streaming-kafka library has appropriate transitive dependencies.
Either version of the spark-streaming-kafka library should work with 1.0 brokers; what problems were you having? On Mon, Dec 25, 2017 at 7:58 PM, Diogo Munaro Vieira <diogo.mun...@corp.globo.com> wrote: > Hey Serkan, it depends of your Kafka version... Is it 0.8.2? > > Em 25 de dez de 2017 06:17, "Serkan TAS" <serkan....@enerjisa.com> escreveu: >> >> Hi, >> >> >> >> Working on spark 2.2.0 cluster and 1.0 kafka brokers. >> >> >> >> I was using the library >> >> "org.apache.spark" % "spark-streaming-kafka-0-10_2.11" % "2.2.0" >> >> >> >> and had lots of problems during streaming process then downgraded to >> >> "org.apache.spark" % "spark-streaming-kafka-0-8_2.11" % >> "2.2.0" >> >> >> >> And i know there is also another path which is using kafka-clients jars >> which has the latest version of 1.0.0 >> >> >> >> <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients --> >> >> <dependency> >> >> <groupId>org.apache.kafka</groupId> >> >> <artifactId>kafka-clients</artifactId> >> >> <version>1.0.0</version> >> >> </dependency> >> >> >> >> I am confused which path is the right one >> >> >> >> Thanks… >> >> >> >> >> >> >> ________________________________ >> >> Bu ileti hukuken korunmuş, gizli veya ifşa edilmemesi gereken bilgiler >> içerebilir. Şayet mesajın gönderildiği kişi değilseniz, bu iletiyi çoğaltmak >> ve dağıtmak yasaktır. Bu mesajı yanlışlıkla alan kişi, bu durumu derhal >> gönderene telefonla ya da e-posta ile bildirmeli ve bilgisayarından >> silmelidir. Bu iletinin içeriğinden yalnızca iletiyi gönderen kişi >> sorumludur. >> >> This communication may contain information that is legally privileged, >> confidential or exempt from disclosure. If you are not the intended >> recipient, please note that any dissemination, distribution, or copying of >> this communication is strictly prohibited. Anyone who receives this message >> in error should notify the sender immediately by telephone or by return >> communication and delete it from his or her computer. Only the person who >> has sent this message is responsible for its content. --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
