Re: Can's create Kafka stream in spark shell

This is how you deal with deduplicate errors: libraryDependencies ++= Seq( ("org.apache.spark" % "spark-streaming_2.10" % "1.1.0" % "provided"). *exclude("org.eclipse.jetty.orbit", "javax.transaction").* *exclude("org.eclipse.jetty.orbit", "javax.mail").* *exclude("org.eclipse.jetty.

Re: Can's create Kafka stream in spark shell

Same error. I saw someone reported the same issue, e.g. http://apache-spark-user-list.1001560.n3.nabble.com/Spark-streaming-kafka-error-td9106.html Should I use "sbt assembly"? It failed for deduplicate though. error] (*:assembly) deduplicate: different file contents found in the following: [erro

Re: Can's create Kafka stream in spark shell

Can you try: sbt: name := "Simple Project" version := "1.1" scalaVersion := "2.10.4" libraryDependencies ++= Seq( "org.apache.spark" %% "spark-core" % "1.1.0", "org.apache.spark" %% "spark-streaming" % "1.1.0", "org.apache.spark" %% "spark-streaming-kafka" % "1.1.0" ) Tha

Re: Can's create Kafka stream in spark shell

Thanks Akhil. I tried spark-submit and saw the same issue. I double checked the versions and they look ok. Are you seeing any obvious issues? sbt: name := "Simple Project" version := "1.1" scalaVersion := "2.10.4" libraryDependencies ++= Seq( "org.apache.spark" %% "spark-core" % "1.1.0

Re: Can's create Kafka stream in spark shell

Just make sure you have the same version of spark-streaming-kafka jar and spark in your classpath. Thanks Best Regards On Tue, Oct 14, 2014 at 9:02 AM, Gary Zhao wrote: > Hello > > I'm trying to connect kafka in spa

Can's create Kafka stream in spark shell

Hello I'm trying to connect kafka in spark shell, but failed as below. Could you take a look what I missed. scala> val kafkaStream = KafkaUtils.createStream(ssc, "test-vip.snc1:2181", "test_spark", Map("user-test"->1)) error: bad symbolic reference. A signature in KafkaUtils.class refers to term