Re: Trident Remote DRPC Possible?

[Cheney Chen]

Anyone has great idea here?

On Tue, Oct 11, 2016 at 11:05 PM, Cheney Chen  wrote:

> Hi there,
>
> I tried out Local DRPC similar to the example https://github.com/
> apache/storm/blob/0a5a069c6223bdb1e7b9318f208c1b29cb302560/examples/storm-
> starter/src/jvm/org/apache/storm/starter/trident/
> TridentKafkaWordCount.java
>
> It's working well. Eventually I need to query state remotely from hosts
> outside storm cluster.
> But not able to find any example after checking through storm code so far.
> Only local drpc is supported by trident?
>
> Anyone has great example that can work out?
>
>
> --
> Regards,
> Qili Chen (Cheney)
>
> E-mail: tbcql1...@gmail.com
> MP: (+1) 4086217503
>



-- 
Regards,
Qili Chen (Cheney)

E-mail: tbcql1...@gmail.com
MP: (+1) 4086217503

Re: Storm-related interview on Linux.com

[Jungtaek Lim]

Nice interview, Julien!

I think crawler is the interesting use case of Apache Storm which other
declarative style streaming frameworks are hard to get.
(I wrote the non-general one last year for internal project, too.)

Hope that your session will be crowded. Thanks for sharing.

- Jungtaek Lim (HeartSaVioR)

2016년 10월 13일 (목) 오후 4:07, Ankur Garg 님이 작성:

> Thanks Julien for sharing this  and all the best for the conference :).
>
> On Wed, 12 Oct 2016 at 22:24 Julien Nioche 
> wrote:
>
> Hi,
>
> FYI, Linux.com have just published an interview [1] I gave on StormCrawler
>  in advance of the presentation I'll be giving
> at ApacheCon [2] in November.
>
> Anyone going to ApacheCon in the Storm community, feel free to find me
> there and have a chat.
>
> Have a nice day
>
> Julien
>
> [1]
> https://www.linux.com/news/stormcrawler-open-source-sdk-building-web-crawlers-apachestorm
>
> [2]
> https://apachebigdataeu2016.sched.org/event/8Tzb/low-latency-web-crawling-on-apache-storm
>
> --
>
> *Open Source Solutions for Text Engineering*
>
> http://www.digitalpebble.com
> http://digitalpebble.blogspot.com/
> #digitalpebble 
>
>

Re: Kafka-Storm spout error: java.lang.NoSuchMethodError: org.apache.kafka.clients.consumer.KafkaConsumer.subscribe

[Jungtaek Lim]

Sorry to visit lately.

storm-kafka-client 1.0.x is guaranteed to work with Kafka 0.9.x, and 1.1.0
is guaranteed to work with Kafka 0.10.0.
Since Apache Storm 1.1.0 is not release yet, you may need to build custom
version for yourself. 1.x-branch is for 1.1.0.

You can also use storm-kafka for Kafka 0.10.0 instead, and move to
storm-kafka-client once Storm 1.1.0 is released.

Hope it helps.

Thanks,
Jungtaek Lim (HeartSaVioR)


2016년 10월 6일 (목) 오후 11:39, shahab 님이 작성:

> Hi.
>
> I am using  storm version 1.0.2  and kafka client version 0.10.0.0.
> When I run the new kafka-spout example in
> https://github.com/apache/storm/tree/master/external/storm-kafka-client/src/test/java/org/apache/storm/kafka/spout/test
>  I am getting the following exception, which I have no idea where it comes
> from.
> I would appreciate if anyone  has nay experience in this regard,
>
> best,
> Shahab
> Here is stack trace;
> java.lang.NoSuchMethodError:
>
> org.apache.kafka.clients.consumer.KafkaConsumer.subscribe(Ljava/util/List;Lorg/apache/kafka/clients/consumer/ConsumerRebalanceListener;)V
> at
> org.apache.storm.kafka.spout.KafkaSpout.subscribeKafkaConsumer(KafkaSpout.java:350)
> ~[storm-kafka-client-1.0.2.jar:1.0.2]
> at org.apache.storm.kafka.spout.KafkaSpout.activate(KafkaSpout.java:341)
> ~[storm-kafka-client-1.0.2.jar:1.0.2]
> at
> org.apache.storm.daemon.executor$fn__7990$fn__8005$fn__8036.invoke(executor.clj:643)
> ~[storm-core-1.0.2.jar:1.0.2]
> at org.apache.storm.util$async_loop$fn__624.invoke(util.clj:484)
> [storm-core-1.0.2.jar:1.0.2]
> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_74]
> 29968 [Thread-22-kafka_spout-executor[4 4]] ERROR o.a.s.d.executor -
> java.lang.NoSuchMethodError:
> org.apache.kafka.clients.consumer.KafkaConsumer.subscribe(Ljava/util/List;Lorg/apache/kafka/clients/consumer/ConsumerRebalanceListener;)V
> at
> org.apache.storm.kafka.spout.KafkaSpout.subscribeKafkaConsumer(KafkaSpout.java:350)
> ~[storm-kafka-client-1.0.2.jar:1.0.2]
> at org.apache.storm.kafka.spout.KafkaSpout.activate(KafkaSpout.java:341)
> ~[storm-kafka-client-1.0.2.jar:1.0.2]
> at
> org.apache.storm.daemon.executor$fn__7990$fn__8005$fn__8036.invoke(executor.clj:643)
> ~[storm-core-1.0.2.jar:1.0.2]
> at org.apache.storm.util$async_loop$fn__624.invoke(util.clj:484)
> [storm-core-1.0.2.jar:1.0.2]
> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_74]
> 29997 [Thread-22-kafka_spout-executor[4 4]] ERROR o.a.s.util - Halting
> process: ("Worker died")
> java.lang.RuntimeException: ("Worker died")
> at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341)
> [storm-core-1.0.2.jar:1.0.2]
> at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.7.0.jar:?]
> at org.apache.storm.daemon.worker$fn__8659$fn__8660.invoke(worker.clj:761)
> [storm-core-1.0.2.jar:1.0.2]
> at
> org.apache.storm.daemon.executor$mk_executor_data$fn__7875$fn__7876.invoke(executor.clj:274)
> [storm-core-1.0.2.jar:1.0.2]
> at org.apache.storm.util$async_loop$fn__624.invoke(util.clj:494)
> [storm-core-1.0.2.jar:1.0.2]
> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_74]
>
> Process finished with exit code 1
>
>