Hi Sajith, Have you tried building storm-0mq from [1] and copying the jar file into STORM_HOME/lib?
You might find this thread [2] useful as well. [1] https://github.com/ptgoetz/storm-0mq [2] http://mail-archives.apache.org/mod_mbox/storm-user/201404.mbox/%3CCALFqTqTS5t8=igkpcjs+b96vwvcdsfkesv++ym6osa6bph4...@mail.gmail.com%3E Thanks, Lasantha On Mon, May 5, 2014 at 5:32 PM, Sajith <[email protected]> wrote: > Hi Tbot, > > Thank you very much for the response. I got this working with netty, but > actually i want to switch from netty to ZeroMQ. I came across this error > when i tried to do so. > > Thanks, > Sajith. > > > On Mon, May 5, 2014 at 5:28 PM, Ebot Tabi <[email protected]> wrote: > >> Hi Sajith, >> >> Please kindly find this configs for storm netty transport layer below, >> which works for me very well. >> >> >> worker.childopts: "-Xmx2048m -XX:+UseConcMarkSweepGC -XX:+UseParNewGC >> -XX:+UseConcMarkSweepGC -XX:NewSize=128m >> -XX:CMSInitiatingOccupancyFraction=70 -XX:-CMSConcurrentMTEnabled >> -Djava.net.preferIPv4Stack=true" >> >> supervisor.childopts: "-Xmx256m" >> >> nimbus.childopts: "-Xmx1024m" >> >> ui.childopts: "-Xmx768m" >> >> nimbus.thrift.threads: 256 >> >> >> >> ########### These MUST be filled in for a storm configuration >> >> storm.zookeeper.servers: >> >> - "192.x.x.x" >> >> # - "server2" >> >> # >> >> nimbus.host: "192.168.56.10" >> >> # >> >> # >> >> # ##### These may optionally be filled in: >> >> # >> >> ## List of custom serializations >> >> # topology.kryo.register: >> >> # - org.mycompany.MyType >> >> # - org.mycompany.MyType2: org.mycompany.MyType2Serializer >> >> # >> >> ## List of custom kryo decorators >> >> # topology.kryo.decorators: >> >> # - org.mycompany.MyDecorator >> >> # >> >> ## Locations of the drpc servers >> >> drpc.servers: >> >> - "192.x.x.x" >> >> # - "server2" >> >> supervisor.slots.ports: >> >> - 6700 >> >> - 6701 >> >> - 6702 >> >> - 6703 >> >> >> ui.port: 8080 >> >> storm.local.dir: "/var/storm" >> >> >> >> storm.messaging.transport: "backtype.storm.messaging.netty.Context" >> >> storm.messaging.netty.server_worker_threads: 1 >> >> storm.messaging.netty.client_worker_threads: 1 >> >> storm.messaging.netty.buffer_size: 5242880 >> >> storm.messaging.netty.max_retries: 100 >> >> storm.messaging.netty.max_wait_ms: 1000 >> >> storm.messaging.netty.min_wait_ms: 100 >> >> >> >> >> On Mon, May 5, 2014 at 9:58 AM, Sajith <[email protected]> wrote: >> >>> Hi all, >>> >>> I'm trying to switch from netty yo ZeroMQ and my storm version is >>> 0.9.2-SNAPSHOT. I copied zmq.jar to STORM_HOME/lib and the necessary libs >>> into /usr/local/lib. Then I sent storm.messaging.transport: >>> "backtype.storm.messaging.zmq" in storm.yaml. But i still get the >>> following error when I submit a topology, >>> >>> 2014-05-05 01:58:41 b.s.d.worker [ERROR] Error on initialization of >>> server mk-worker >>> java.lang.RuntimeException: Fail to construct messaging plugin from >>> plugin backtype.storm.messaging.zmq >>> at >>> backtype.storm.messaging.TransportFactory.makeContext(TransportFactory.java:53) >>> ~[storm-core-0.9.2-incubating-SNAPSHOT.jar:0.9.2-incubating-SNAPSHOT] >>> at >>> backtype.storm.daemon.worker$worker_data$fn__4263.invoke(worker.clj:185) >>> ~[na:0.9.2-incubating-SNAPSHOT] >>> at backtype.storm.util$assoc_apply_self.invoke(util.clj:792) >>> ~[storm-core-0.9.2-incubating-SNAPSHOT.jar:0.9.2-incubating-SNAPSHOT] >>> at backtype.storm.daemon.worker$worker_data.invoke(worker.clj:181) >>> ~[storm-core-0.9.2-incubating-SNAPSHOT.jar:0.9.2-incubating-SNAPSHOT] >>> at >>> backtype.storm.daemon.worker$eval4413$exec_fn__1102__auto____4414.invoke(worker.clj:353) >>> ~[na:0.9.2-incubating-SNAPSHOT] >>> at clojure.lang.AFn.applyToHelper(AFn.java:185) >>> [clojure-1.4.0.jar:na] >>> at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.4.0.jar:na] >>> at clojure.core$apply.invoke(core.clj:601) ~[clojure-1.4.0.jar:na] >>> at >>> backtype.storm.daemon.worker$eval4413$mk_worker__4469.doInvoke(worker.clj:344) >>> [na:0.9.2-incubating-SNAPSHOT] >>> at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.4.0.jar:na] >>> at backtype.storm.daemon.worker$_main.invoke(worker.clj:454) >>> [na:0.9.2-incubating-SNAPSHOT] >>> at clojure.lang.AFn.applyToHelper(AFn.java:172) >>> [clojure-1.4.0.jar:na] >>> at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.4.0.jar:na] >>> at backtype.storm.daemon.worker.main(Unknown Source) >>> [storm-core-0.9.2-incubating-SNAPSHOT.jar:0.9.2-incubating-SNAPSHOT] >>> Caused by: java.lang.ClassNotFoundException: backtype.storm.messaging.zmq >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366) >>> ~[na:1.7.0_45] >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>> ~[na:1.7.0_45] >>> at java.security.AccessController.doPrivileged(Native Method) >>> ~[na:1.7.0_45] >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>> ~[na:1.7.0_45] >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) >>> ~[na:1.7.0_45] >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>> ~[na:1.7.0_45] >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) >>> ~[na:1.7.0_45] >>> at java.lang.Class.forName0(Native Method) ~[na:1.7.0_45] >>> at java.lang.Class.forName(Class.java:190) ~[na:1.7.0_45] >>> at >>> backtype.storm.messaging.TransportFactory.makeContext(TransportFactory.java:38) >>> ~[storm-core-0.9.2-incubating-SNAPSHOT.jar:0.9.2-incubating-SNAPSHOT] >>> ... 13 common frames omitted >>> 2014-05-05 01:58:41 b.s.util [INFO] Halting process: ("Error on >>> initialization") >>> >>> Can someone please tell me what have gone wrong or point me to a >>> reference on the correct way for switching the transport. >>> >>> Thanks, >>> Sajith. >>> >> >> >> >> -- >> Ebot T. >> >> >
