Will give it a shot, later. BTW, this forced me to move to Scala! Decided to
design our aggregation frame-work in scala for now.
On 07-Mar-2014, at 6:02 AM, Tathagata Das <tathagata.das1...@gmail.com> wrote:
> I dont have a Eclipse setup so I am not sure what is going on here. I would
> try to use maven in the command line with a pom to see if this compiles.
> Also, try to cleanup your system maven cache. Who knows if it had pulled in a
> wrong version of kafka 0.8 and using it all the time. Blowing away the cache
> and clean compiling will make sure the right kafka will be loaded.
>
> Hope this helps.
>
> TD
>
>
> On Sat, Mar 1, 2014 at 8:26 PM, venki-kratos <ve...@thekratos.com> wrote:
> I am trying to user code similar to following :
> public JavaPairDStream<String, String> openStream() {
> HashMap<String, String> kafkaParams = Maps.newHashMap();
> kafkaParams.put(ZK_CONNECT,kafkaConfig.getString(ZK_CONNECT));
>
> kafkaParams.put(CONSUMER_GRP_ID,kafkaConfig.getString(CONSUMER_GRP_ID));
>
> Map<String,Integer> topicMap = Maps.newHashMap();
> topicMap.put(kafkaConfig.getString(ZK_TOPIC),
> kafkaConfig.getInteger(CONSUMER_THREAD_COUNT, 1));
> JavaPairDStream<String, String> inputStream =
> KafkaUtils.createStream(streamingContext,
> String.class, String.class,
> StringDecoder.class,
> StringDecoder.class,
> kafkaParams,
> topicMap, StorageLevel.MEMORY_AND_DISK_SER_2());
> return inputStream;
> }
>
> I have spark-streaming_2.10-0.9.0-incubating.jar and
> spark-streaming-kafka_2.10-0.9.0-incubating.jar
> in the classpath using POM and m2e in Eclipse. JVM version is set to 1.6
>
> I get the following error,
>
> 14/03/02 09:29:15 INFO kafka.KafkaReceiver: Connected to localhost:2181
> 14/03/02 09:29:15 ERROR kafka.KafkaReceiver: Error receiving data
> java.lang.NoSuchMethodException:
> java.lang.Object.<init>(kafka.utils.VerifiableProperties)
> at java.lang.Class.getConstructor0(Class.java:2763)
> at java.lang.Class.getConstructor(Class.java:1693)
> at
> org.apache.spark.streaming.kafka.KafkaReceiver.onStart(KafkaInputDStream.scala:108)
> at
> org.apache.spark.streaming.dstream.NetworkReceiver.start(NetworkInputDStream.scala:126)
> at
> org.apache.spark.streaming.scheduler.NetworkInputTracker$ReceiverExecutor$$anonfun$8.apply(NetworkInputTracker.scala:173)
> at
> org.apache.spark.streaming.scheduler.NetworkInputTracker$ReceiverExecutor$$anonfun$8.apply(NetworkInputTracker.scala:169)
> at
> org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:884)
> at
> org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:884)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:109)
> at org.apache.spark.scheduler.Task.run(Task.scala:53)
> at .....
>
> This is similar to code in JavaKafkaStreamSuite.testKafkaStream. I find that
> the kafka jar - kafka_2.10-0.8.0 does have such a constructor.
>
> What is going wrong? Can someone help solve this mystery and help with my
> misery? Basically stuck for last 2 days - as I am a Java Guy and would like
> to develop downstream code in Java
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/NoSuchMethodError-in-KafkaReciever-tp2209.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
