Hi Sajith,
Please kindly find this configs for storm netty transport layer below,
which works for me very well.
worker.childopts: "-Xmx2048m -XX:+UseConcMarkSweepGC -XX:+UseParNewGC
-XX:+UseConcMarkSweepGC -XX:NewSize=128m
-XX:CMSInitiatingOccupancyFraction=70 -XX:-CMSConcurrentMTEnabled
-Djava.net.preferIPv4Stack=true"
supervisor.childopts: "-Xmx256m"
nimbus.childopts: "-Xmx1024m"
ui.childopts: "-Xmx768m"
nimbus.thrift.threads: 256
########### These MUST be filled in for a storm configuration
storm.zookeeper.servers:
- "192.x.x.x"
# - "server2"
#
nimbus.host: "192.168.56.10"
#
#
# ##### These may optionally be filled in:
#
## List of custom serializations
# topology.kryo.register:
# - org.mycompany.MyType
# - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
# - org.mycompany.MyDecorator
#
## Locations of the drpc servers
drpc.servers:
- "192.x.x.x"
# - "server2"
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
ui.port: 8080
storm.local.dir: "/var/storm"
storm.messaging.transport: "backtype.storm.messaging.netty.Context"
storm.messaging.netty.server_worker_threads: 1
storm.messaging.netty.client_worker_threads: 1
storm.messaging.netty.buffer_size: 5242880
storm.messaging.netty.max_retries: 100
storm.messaging.netty.max_wait_ms: 1000
storm.messaging.netty.min_wait_ms: 100
On Mon, May 5, 2014 at 9:58 AM, Sajith <[email protected]> wrote:
> Hi all,
>
> I'm trying to switch from netty yo ZeroMQ and my storm version is
> 0.9.2-SNAPSHOT. I copied zmq.jar to STORM_HOME/lib and the necessary libs
> into /usr/local/lib. Then I sent storm.messaging.transport:
> "backtype.storm.messaging.zmq" in storm.yaml. But i still get the
> following error when I submit a topology,
>
> 2014-05-05 01:58:41 b.s.d.worker [ERROR] Error on initialization of server
> mk-worker
> java.lang.RuntimeException: Fail to construct messaging plugin from plugin
> backtype.storm.messaging.zmq
> at
> backtype.storm.messaging.TransportFactory.makeContext(TransportFactory.java:53)
> ~[storm-core-0.9.2-incubating-SNAPSHOT.jar:0.9.2-incubating-SNAPSHOT]
> at
> backtype.storm.daemon.worker$worker_data$fn__4263.invoke(worker.clj:185)
> ~[na:0.9.2-incubating-SNAPSHOT]
> at backtype.storm.util$assoc_apply_self.invoke(util.clj:792)
> ~[storm-core-0.9.2-incubating-SNAPSHOT.jar:0.9.2-incubating-SNAPSHOT]
> at backtype.storm.daemon.worker$worker_data.invoke(worker.clj:181)
> ~[storm-core-0.9.2-incubating-SNAPSHOT.jar:0.9.2-incubating-SNAPSHOT]
> at
> backtype.storm.daemon.worker$eval4413$exec_fn__1102__auto____4414.invoke(worker.clj:353)
> ~[na:0.9.2-incubating-SNAPSHOT]
> at clojure.lang.AFn.applyToHelper(AFn.java:185) [clojure-1.4.0.jar:na]
> at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.4.0.jar:na]
> at clojure.core$apply.invoke(core.clj:601) ~[clojure-1.4.0.jar:na]
> at
> backtype.storm.daemon.worker$eval4413$mk_worker__4469.doInvoke(worker.clj:344)
> [na:0.9.2-incubating-SNAPSHOT]
> at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.4.0.jar:na]
> at backtype.storm.daemon.worker$_main.invoke(worker.clj:454)
> [na:0.9.2-incubating-SNAPSHOT]
> at clojure.lang.AFn.applyToHelper(AFn.java:172) [clojure-1.4.0.jar:na]
> at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.4.0.jar:na]
> at backtype.storm.daemon.worker.main(Unknown Source)
> [storm-core-0.9.2-incubating-SNAPSHOT.jar:0.9.2-incubating-SNAPSHOT]
> Caused by: java.lang.ClassNotFoundException: backtype.storm.messaging.zmq
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> ~[na:1.7.0_45]
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> ~[na:1.7.0_45]
> at java.security.AccessController.doPrivileged(Native Method)
> ~[na:1.7.0_45]
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> ~[na:1.7.0_45]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_45]
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> ~[na:1.7.0_45]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_45]
> at java.lang.Class.forName0(Native Method) ~[na:1.7.0_45]
> at java.lang.Class.forName(Class.java:190) ~[na:1.7.0_45]
> at
> backtype.storm.messaging.TransportFactory.makeContext(TransportFactory.java:38)
> ~[storm-core-0.9.2-incubating-SNAPSHOT.jar:0.9.2-incubating-SNAPSHOT]
> ... 13 common frames omitted
> 2014-05-05 01:58:41 b.s.util [INFO] Halting process: ("Error on
> initialization")
>
> Can someone please tell me what have gone wrong or point me to a reference
> on the correct way for switching the transport.
>
> Thanks,
> Sajith.
>
--
Ebot T.
