Pavan,
It would be helpful if you can post your storm.yaml.
Make sure user "storm" has permissions to your storm
installation dir. and you used the same storm config on all
your machines.
-Harsha
On Tue, Sep 2, 2014, at 06:53 AM, Supun Kamburugamuva wrote:
Hi Pavan,
It seems you have a permission issue. Please check weather the
directories that the storm user has appropriate permissions on
the directories that contain the storm jars.
Thanks,
Supun..
On Tue, Sep 2, 2014 at 9:13 AM, Pavan Jakati G
<[1][email protected]> wrote:
Can anybody help us get rid of below error :
sudo -u storm /usr/java/jdk1.7.0_65/bin/java -server -Xmx768m
-Djava.library.path=storm-local/supervisor/stormdist/PaVan-14-1
409661199/resources/Linux-amd64:storm-local/supervisor/stormdis
t/PaVan-14-1409661199/resources:/usr/local/lib:/opt/local/lib:/
usr/lib -Dlogfile.name=worker-6703.log
-Dstorm.home=/root/apache-storm-0.9.2-incubating
-Dlogback.configurationFile=/root/apache-storm-0.9.2-incubating
/logback/cluster.xml -Dstorm.id=PaVan-14-1409661199
-Dworker.id=156a8af9-fa3b-4772-b91c-787490fe0b34
-Dworker.port=6703 -cp
/root/apache-storm-0.9.2-incubating/lib/json-simple-1.1.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/
root/apache-storm-0.9.2-incubating/lib/asm-4.0.jar:/root/apache
-storm-0.9.2-incubating/lib/commons-io-2.4.jar:/root/apache-sto
rm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/root/apache-storm
-0.9.2-incubating/lib/jline-2.11.jar:/root/apache-storm-0.9.2-i
ncubating/lib/tools.logging-0.2.3.jar:/root/apache-storm-0.9.2-
incubating/lib/logback-classic-1.0.6.jar:/root/apache-storm-0.9
.2-incubating/lib/commons-logging-1.1.3.jar:/root/apache-storm-
0.9.2-incubating/lib/ring-core-1.1.5.jar:/root/apache-storm-0.9
.2-incubating/lib/ring-devel-0.3.11.jar:/root/apache-storm-0.9.
2-incubating/lib/curator-client-2.4.0.jar:/root/apache-storm-0.
9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/root/apache-storm-
0.9.2-incubating/lib/clj-time-0.4.1.jar:/root/apache-storm-0.9.
2-incubating/lib/commons-lang-2.5.jar:/root/apache-storm-0.9.2-
incubating/lib/zookeeper-3.4.5.jar:/root/apache-storm-0.9.2-inc
ubating/lib/compojure-1.1.3.jar:/root/apache-storm-0.9.2-incuba
ting/lib/joda-time-2.0.jar:/root/apache-storm-0.9.2-incubating/
lib/chill-java-0.3.5.jar:/root/apache-storm-0.9.2-incubating/li
b/clout-1.0.1.jar:/root/apache-storm-0.9.2-incubating/lib/kryo-
2.21.jar:/root/apache-storm-0.9.2-incubating/lib/snakeyaml-1.11
.jar:/root/apache-storm-0.9.2-incubating/lib/minlog-1.2.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/storm-core-0.9.2-incubatin
g.jar:/root/apache-storm-0.9.2-incubating/lib/jgrapht-core-0.9.
0.jar:/root/apache-storm-0.9.2-incubating/lib/slf4j-api-1.6.5.j
ar:/root/apache-storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/roo
t/apache-storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar
:/root/apache-storm-0.9.2-incubating/lib/guava-13.0.jar:/root/a
pache-storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.j
ar:/root/apache-storm-0.9.2-incubating/lib/servlet-api-2.5.jar:
/root/apache-storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.
jar:/root/apache-storm-0.9.2-incubating/lib/jetty-util-6.1.26.j
ar:/root/apache-storm-0.9.2-incubating/lib/objenesis-1.2.jar:/r
oot/apache-storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/root
/apache-storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.ja
r:/root/apache-storm-0.9.2-incubating/lib/commons-codec-1.6.jar
:/root/apache-storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/roo
t/apache-storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/root
/apache-storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar
:/root/apache-storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/r
oot/apache-storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/roo
t/apache-storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/root/
apache-storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/root/a
pache-storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/root/apache-
storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/root/apache-stor
m-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/root/apach
e-storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/root/apac
he-storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/root/apac
he-storm-0.9.2-incubating/conf:storm-local/supervisor/stormdist
/PaVan-14-1409661199/stormjar.jar backtype.storm.daemon.worker
PaVan-14-1409661199 0574446a-f73d-42b0-bcc7-e6dd449cb75a 6703
156a8af9-fa3b-4772-b91c-787490fe0b34
Error: Could not find or load main class
backtype.storm.daemon.worker
Regards,
PaVan…
From: Pavan Jakati G
Sent: 02 September 2014 15:23
To: [2][email protected]
Subject: RE: REMOTE MODE STORM DEV
sudo -u storm '/usr/java/jdk1.7.0_65/bin/java' '-server'
'-Xmx768m' '-Djava.library.path=storm-local/sup
ervisor/stormdist/PaVan-10-1409648607/resources/Linux-amd64:sto
rm-local/supervisor/stormdist/PaVan-10-1409648607/resources:/us
r/local/lib:/opt/local/lib:/usr/lib'
'-Dlogfile.name=worker-6701.log'
'-Dstorm.home=/root/apache-storm-0.9.2-incubating'
'-Dlogback.configurationFile=/root/apache-storm-0.9.2-incubatin
g/logback/cluster.xml' '-Dstorm.id=PaVan-10-1409648607'
'-Dworker.id=b89ee630-7bd6-4976-8050-c43644c028ed'
'-Dworker.port=6701' '-cp'
'/root/apache-storm-0.9.2-incubating/lib/json-simple-1.1.jar:/r
oot/apache-storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:
/root/apache-storm-0.9.2-incubating/lib/asm-4.0.jar:/root/apach
e-storm-0.9.2-incubating/lib/commons-io-2.4.jar:/root/apache-st
orm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/root/apache-stor
m-0.9.2-incubating/lib/jline-2.11.jar:/root/apache-storm-0.9.2-
incubating/lib/tools.logging-0.2.3.jar:/root/apache-storm-0.9.2
-incubating/lib/logback-classic-1.0.6.jar:/root/apache-storm-0.
9.2-incubating/lib/commons-logging-1.1.3.jar:/root/apache-storm
-0.9.2-incubating/lib/ring-core-1.1.5.jar:/root/apache-storm-0.
9.2-incubating/lib/ring-devel-0.3.11.jar:/root/apache-storm-0.9
.2-incubating/lib/curator-client-2.4.0.jar:/root/apache-storm-0
.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/root/apache-storm
-0.9.2-incubating/lib/clj-time-0.4.1.jar:/root/apache-storm-0.9
.2-incubating/lib/commons-lang-2.5.jar:/root/apache-storm-0.9.2
-incubating/lib/zookeeper-3.4.5.jar:/root/apache-storm-0.9.2-in
cubating/lib/compojure-1.1.3.jar:/root/apache-storm-0.9.2-incub
ating/lib/joda-time-2.0.jar:/root/apache-storm-0.9.2-incubating
/lib/chill-java-0.3.5.jar:/root/apache-storm-0.9.2-incubating/l
ib/clout-1.0.1.jar:/root/apache-storm-0.9.2-incubating/lib/kryo
-2.21.jar:/root/apache-storm-0.9.2-incubating/lib/snakeyaml-1.1
1.jar:/root/apache-storm-0.9.2-incubating/lib/minlog-1.2.jar:/r
oot/apache-storm-0.9.2-incubating/lib/storm-core-0.9.2-incubati
ng.jar:/root/apache-storm-0.9.2-incubating/lib/jgrapht-core-0.9
.0.jar:/root/apache-storm-0.9.2-incubating/lib/slf4j-api-1.6.5.
jar:/root/apache-storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/r
oot/apache-storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/curator-framework-2.4.0.ja
r:/root/apache-storm-0.9.2-incubating/lib/guava-13.0.jar:/root/
apache-storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/r
oot/apache-storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.
jar:/root/apache-storm-0.9.2-incubating/lib/servlet-api-2.5.jar
:/root/apache-storm-0.9.2-incubating/lib/reflectasm-1.07-shaded
.jar:/root/apache-storm-0.9.2-incubating/lib/jetty-util-6.1.26.
jar:/root/apache-storm-0.9.2-incubating/lib/objenesis-1.2.jar:/
root/apache-storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/roo
t/apache-storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.j
ar:/root/apache-storm-0.9.2-incubating/lib/commons-codec-1.6.ja
r:/root/apache-storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/roo
t/apache-storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.ja
r:/root/apache-storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/
root/apache-storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/root
/apache-storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/root/
apache-storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/root/apache
-storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/root/apache-sto
rm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/root/apac
he-storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/root/apa
che-storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/root/apa
che-storm-0.9.2-incubating/conf:storm-local/supervisor/stormdis
t/PaVan-10-1409648607/stormjar.jar'
'backtype.storm.daemon.worker' 'PaVan-10-1409648607'
'0574446a-f73d-42b0-bcc7-e6dd449cb75a' '6701'
'b89ee630-7bd6-4976-8050-c43644c028ed'
Error: Could not find or load main class
backtype.storm.daemon.worker
Regards,
PaVan…
From: Vikas Agarwal [[3]mailto:[email protected]]
Sent: 02 September 2014 15:05
To: [4][email protected]
Subject: Re: REMOTE MODE STORM DEV
Pavan,
Have you tried running the command for launching worker as
logged in supervisor.log? You need to run the command with
storm user and may need to kill existing one if running on same
port.
On Tue, Sep 2, 2014 at 3:02 PM, Pavan Jakati G
<[5][email protected]> wrote:
Is supervisor running or dead ? . Spouts and bolts which run
fine in local mode fails to do same in remote mode. Pls help us
here.
less supervisor.log
m-0.9.2-incubating/lib/commons-io-2.4.jar:/root/apache-storm-0.
9.2-incubating/lib/httpclient-4.3.3.jar:/root/apache-storm-0.9.
2-incubating/lib/jline-2.11.jar:/root/apache-storm-0.9.2-incuba
ting/lib/tools.logging-0.2.3.jar:/root/apache-storm-0.9.2-incub
ating/lib/logback-classic-1.0.6.jar:/root/apache-storm-0.9.2-in
cubating/lib/commons-logging-1.1.3.jar:/root/apache-storm-0.9.2
-incubating/lib/ring-core-1.1.5.jar:/root/apache-storm-0.9.2-in
cubating/lib/ring-devel-0.3.11.jar:/root/apache-storm-0.9.2-inc
ubating/lib/curator-client-2.4.0.jar:/root/apache-storm-0.9.2-i
ncubating/lib/clj-stacktrace-0.2.4.jar:/root/apache-storm-0.9.2
-incubating/lib/clj-time-0.4.1.jar:/root/apache-storm-0.9.2-inc
ubating/lib/commons-lang-2.5.jar:/root/apache-storm-0.9.2-incub
ating/lib/zookeeper-3.4.5.jar:/root/apache-storm-0.9.2-incubati
ng/lib/compojure-1.1.3.jar:/root/apache-storm-0.9.2-incubating/
lib/joda-time-2.0.jar:/root/apache-storm-0.9.2-incubating/lib/c
hill-java-0.3.5.jar:/root/apache-storm-0.9.2-incubating/lib/clo
ut-1.0.1.jar:/root/apache-storm-0.9.2-incubating/lib/kryo-2.21.
jar:/root/apache-storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:
/root/apache-storm-0.9.2-incubating/lib/minlog-1.2.jar:/root/ap
ache-storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar
:/root/apache-storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar
:/root/apache-storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/r
oot/apache-storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/root/ap
ache-storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/root/apa
che-storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/roo
t/apache-storm-0.9.2-incubating/lib/guava-13.0.jar:/root/apache
-storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/root/ap
ache-storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/r
oot/apache-storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/root
/apache-storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/
root/apache-storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/r
oot/apache-storm-0.9.2-incubating/lib/objenesis-1.2.jar:/root/a
pache-storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/root/apac
he-storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/roo
t/apache-storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/root/apa
che-storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/root/apac
he-storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/roo
t/apache-storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/root/a
pache-storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/root/apa
che-storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/root/apach
e-storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/root/apache
-storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/root/apache-storm
-0.9.2-incubating/lib/httpcore-4.3.2.jar:/root/apache-storm-0.9
.2-incubating/lib/servlet-api-2.5-20081211.jar:/root/apache-sto
rm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/root/apache-st
orm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/root/apache-st
orm-0.9.2-incubating/conf:storm-local/supervisor/stormdist/PRIM
ETopology-6-1409635517/stormjar.jar'
'backtype.storm.daemon.worker' 'PRIMETopology-6-1409635517'
'0574446a-f73d-42b0-bcc7-e6dd449cb75a' '6702'
'ddf2747a-276a-4724-ba08-4f5d46b0b5ec'
2014-09-02 05:34:01 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:02 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:02 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:03 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:03 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:04 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:04 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:05 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:05 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:06 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:06 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:07 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:07 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:08 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:08 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:09 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
Worker Log:
2014-09-02 09:03:37 b.s.d.executor [INFO] Loading executor
SOIMbolt:[1 1]
2014-09-02 09:03:37 b.s.d.executor [INFO] Loaded executor tasks
SOIMbolt:[1 1]
2014-09-02 09:03:37 b.s.d.executor [INFO] Prepared bolt
__system:(-1)
2014-09-02 09:03:37 b.s.d.executor [INFO] Finished loading
executor SOIMbolt:[1 1]
2014-09-02 09:03:37 b.s.d.worker [INFO] Launching
receive-thread for 0574446a-f73d-42b0-bcc7-e6dd449cb75a:6701
2014-09-02 09:03:37 b.s.d.executor [INFO] Preparing bolt
SOIMbolt:(1)
2014-09-02 09:03:37 b.s.d.executor [INFO] Prepared bolt
SOIMbolt:(1)
2014-09-02 09:03:37 b.s.m.n.Server [INFO] Create Netty Server
Netty-server-localhost-6701, buffer_size: 5242880, maxWorkers:
1
2014-09-02 09:03:38 b.s.m.loader [INFO] Starting
receive-thread: [stormId: PaVan-10-1409648607, port: 6701,
thread-id: 0 ]
2014-09-02 09:03:38 b.s.d.worker [INFO] Worker has topology
config {"[6]storm.id" "PaVan-10-1409648607",
"dev.zookeeper.path" "/tmp/dev-storm-zookeeper",
"topology.tick.tuple.freq.secs" nil,
"topology.builtin.metrics.bucket.size.secs" 60,
"topology.fall.back.on.java.serialization" true,
"topology.max.error.report.per.interval" 5, "zmq.linger.millis"
5000, "topology.skip.missing.kryo.registrations" false,
"storm.messaging.netty.client_worker_threads" 1, "ui.childopts"
"-Xmx768m", "storm.zookeeper.session.timeout" 20000,
"nimbus.reassign" true,
"topology.trident.batch.emit.interval.millis" 500,
"drpc.servers" ["192.168.10.7"],
"[7]storm.messaging.netty.flush.check.interval.ms" 10,
"nimbus.monitor.freq.secs" 10, "logviewer.childopts"
"-Xmx128m", "java.library.path"
"/usr/local/lib:/opt/local/lib:/usr/lib",
"topology.executor.send.buffer.size" 1024, "storm.local.dir"
"storm-local", "storm.messaging.netty.buffer_size" 5242880,
"supervisor.worker.start.timeout.secs" 120,
"topology.enable.message.timeouts" true,
"nimbus.cleanup.inbox.freq.secs" 600,
"nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads"
64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host"
"192.168.10.7", "storm.messaging.netty.min_wait_ms" 100,
"storm.zookeeper.port" 2181, "transactional.zookeeper.port"
nil, "topology.executor.receive.buffer.size" 1024,
"transactional.zookeeper.servers" nil, "storm.zookeeper.root"
"/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000,
"supervisor.enable" true,
"storm.messaging.netty.server_worker_threads" 1,
"storm.zookeeper.servers" ["localhost"],
"transactional.zookeeper.root" "/transactional",
"topology.acker.executors" nil, "topology.kryo.decorators" (),
"[8]topology.name" "PaVan", "topology.transfer.buffer.size"
1024, "topology.worker.childopts" nil, "drpc.queue.size" 128,
"worker.childopts" "-Xmx768m",
"supervisor.heartbeat.frequency.secs" 5,
"topology.error.throttle.interval.secs" 10, "zmq.hwm" 0,
"drpc.port" 3772, "supervisor.monitor.frequency.secs" 3,
"drpc.childopts" "-Xmx768m", "topology.receiver.buffer.size" 8,
"task.heartbeat.frequency.secs" 3, "topology.tasks" nil,
"storm.messaging.netty.max_retries" 30,
"topology.spout.wait.strategy"
"backtype.storm.spout.SleepSpoutWaitStrategy",
"nimbus.thrift.max_buffer_size" 1048576,
"topology.max.spout.pending" nil,
"storm.zookeeper.retry.interval" 1000,
"[9]topology.sleep.spout.wait.strategy.time.ms" 1,
"nimbus.topology.validator"
"backtype.storm.nimbus.DefaultTopologyValidator",
"supervisor.slots.ports" [6700 6701 6702 6703],
"topology.debug" false, "nimbus.task.launch.secs" 120,
"nimbus.supervisor.timeout.secs" 60, "topology.kryo.register"
nil, "topology.message.timeout.secs" 30,
"task.refresh.poll.secs" 10, "topology.workers" 1,
"supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627,
"topology.stats.sample.rate" 0.05,
"worker.heartbeat.frequency.secs" 1,
"topology.tuple.serializer"
"backtype.storm.serialization.types.ListDelegateSerializer",
"topology.disruptor.wait.strategy"
"com.lmax.disruptor.BlockingWaitStrategy",
"topology.multilang.serializer"
"backtype.storm.multilang.JsonSerializer",
"nimbus.task.timeout.secs" 30,
"storm.zookeeper.connection.timeout" 15000,
"topology.kryo.factory"
"backtype.storm.serialization.DefaultKryoFactory",
"drpc.invocations.port" 3773, "logviewer.port" 8000,
"zmq.threads" 1, "storm.zookeeper.retry.times" 5,
"topology.worker.receiver.thread.count" 1,
"storm.thrift.transport"
"backtype.storm.security.auth.SimpleTransportPlugin",
"topology.state.synchronization.timeout.secs" 60,
"supervisor.worker.timeout.secs" 30,
"nimbus.file.copy.expiration.secs" 600,
"storm.messaging.transport"
"backtype.storm.messaging.netty.Context",
"[10]logviewer.appender.name" "A1",
"storm.messaging.netty.max_wait_ms" 1000,
"drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false,
"ui.port" 8080, "nimbus.childopts" "-Xmx1024m",
"storm.cluster.mode" "distributed",
"topology.max.task.parallelism" nil,
"storm.messaging.netty.transfer.batch.size" 262144}
2014-09-02 09:03:38 b.s.d.worker [INFO] Worker
b89ee630-7bd6-4976-8050-c43644c028ed for storm
PaVan-10-1409648607 on
0574446a-f73d-42b0-bcc7-e6dd449cb75a:6701 has finished loading
Spout Code :
public class SOIMSpout extends BaseRichSpout
{
private SpoutOutputCollector collector;
boolean _isDistributed;
java.sql.ResultSet SOIMResultset = null;
java.sql.Connection SOIMConnection = null;
java.sql.Statement SOIMStatement = null;
private PreparedStatement SOIMPrepstatement = null;
private PreparedStatement SOIMPrepstatement_Stream =
null;
private PreparedStatement SOIMPrepstatementTotal =
null;
int SOIMTicketno = 0 ;
String SOIMDesc = "", SOIMInf = "", SOIMDate = "",
SOIMDevice = "", SOIMCategory = "", SOIMSubCategory = "",
SOIMPriority="";
String SOIMdbUrl = "", SOIMdbUser = "", SOIMdbPassword
= "", SOIMdbClass="";
/************ MYSQL DB CONNECTION VARIABLES FOR
RECEIVING ************/
public SOIMSpout()
{
SOIMConnector objSOIMConnect = new SOIMConnector();
/********* OBJECT CREATION FOR SOIMConnector CLASS -
SRIDHAR ********/
SOIMdbUrl = objSOIMConnect.dbUrl;
SOIMdbUser = objSOIMConnect.dbUser;
SOIMdbPassword = objSOIMConnect.dbPassword;
}
@Override
public void nextTuple()
{
try
{
SOIMConnection =
DriverManager.getConnection(SOIMdbUrl,SOIMdbUser,SOIMdbPassword
);
//******* Object for jdbc connectivity from
SOIMConnector class -- Sridhar*** //
SOIMStatement =
SOIMConnection.createStatement();
String SOIMQuery = "SELECT
ticketno,description,information,createdon,configitemname,categ
oryname,technologyname,priorityname FROM siom_tosticket";
SOIMResultset =
SOIMStatement.executeQuery(SOIMQuery);
while(SOIMResultset.next())
{
SOIMTicketno =
(SOIMResultset.getInt("ticketno"));
SOIMDesc =
(SOIMResultset.getString("description"));
SOIMInf=
(SOIMResultset.getString("information"));
SOIMDate =
(SOIMResultset.getString("createdon"));
SOIMDevice =
(SOIMResultset.getString("configitemname"));
SOIMCategory =
(SOIMResultset.getString("categoryname"));
SOIMSubCategory =
(SOIMResultset.getString("technologyname"));
SOIMPriority=(SOIMResultset.getString("priorityname"));
//collector.emit(new
Values(SOIMTicketno,SOIMDesc,
SOIMInf,SOIMDate,SOIMDevice,SOIMCategory,SOIMSubCategory,SOIMPr
iority));
//******* Emitting Tuples from
SPOUT to BOLT -- Sridhar*** //
}
collector.emit(new Values(SOIMTicketno,SOIMDesc,
SOIMInf,SOIMDate,SOIMDevice,SOIMCategory,SOIMSubCategory,SOIMPr
iority));
}
catch (Exception e) {
}
finally {
if (SOIMConnection != null) {
@Override
public void ack(Object id)
{
}
@Override
public void fail(Object id)
{
}
@Override
public void declareOutputFields(OutputFieldsDeclarer
declarer)
{
declarer.declare( new Fields("SOIMTicketno","SOIMDesc",
"SOIMInf","SOIMDate","SOIMDevice","SOIMCategory","SOIMSubCatego
ry","SOIMPriority"));
}
@Override
public void close() {
}
@Override
public void activate() {
// TODO Auto-generated method stub
}
@Override
public void deactivate() {
// TODO Auto-generated method stub
}
Regards,
PaVan…
From: Pavan Jakati G
Sent: 02 September 2014 10:56
To: [11][email protected]
Subject: RE: REMOTE MODE STORM DEV
storm jar /root/SOIM/target/SOIMTopology-1.0-SNAPSHOT.jar
Microland.SOIM.SOIMTopology SOIMTopology
Running: /usr/java/jdk1.7.0_65/bin/java -client
-Dstorm.options=
-Dstorm.home=/root/apache-storm-0.9.2-incubating
-Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
-Dstorm.conf.file= -cp
/root/apache-storm-0.9.2-incubating/lib/json-simple-1.1.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/
root/apache-storm-0.9.2-incubating/lib/asm-4.0.jar:/root/apache
-storm-0.9.2-incubating/lib/commons-io-2.4.jar:/root/apache-sto
rm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/root/apache-storm
-0.9.2-incubating/lib/jline-2.11.jar:/root/apache-storm-0.9.2-i
ncubating/lib/tools.logging-0.2.3.jar:/root/apache-storm-0.9.2-
incubating/lib/logback-classic-1.0.6.jar:/root/apache-storm-0.9
.2-incubating/lib/commons-logging-1.1.3.jar:/root/apache-storm-
0.9.2-incubating/lib/ring-core-1.1.5.jar:/root/apache-storm-0.9
.2-incubating/lib/ring-devel-0.3.11.jar:/root/apache-storm-0.9.
2-incubating/lib/curator-client-2.4.0.jar:/root/apache-storm-0.
9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/root/apache-storm-
0.9.2-incubating/lib/clj-time-0.4.1.jar:/root/apache-storm-0.9.
2-incubating/lib/commons-lang-2.5.jar:/root/apache-storm-0.9.2-
incubating/lib/zookeeper-3.4.5.jar:/root/apache-storm-0.9.2-inc
ubating/lib/compojure-1.1.3.jar:/root/apache-storm-0.9.2-incuba
ting/lib/joda-time-2.0.jar:/root/apache-storm-0.9.2-incubating/
lib/chill-java-0.3.5.jar:/root/apache-storm-0.9.2-incubating/li
b/clout-1.0.1.jar:/root/apache-storm-0.9.2-incubating/lib/kryo-
2.21.jar:/root/apache-storm-0.9.2-incubating/lib/snakeyaml-1.11
.jar:/root/apache-storm-0.9.2-incubating/lib/minlog-1.2.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/storm-core-0.9.2-incubatin
g.jar:/root/apache-storm-0.9.2-incubating/lib/jgrapht-core-0.9.
0.jar:/root/apache-storm-0.9.2-incubating/lib/slf4j-api-1.6.5.j
ar:/root/apache-storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/roo
t/apache-storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar
:/root/apache-storm-0.9.2-incubating/lib/guava-13.0.jar:/root/a
pache-storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.j
ar:/root/apache-storm-0.9.2-incubating/lib/servlet-api-2.5.jar:
/root/apache-storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.
jar:/root/apache-storm-0.9.2-incubating/lib/jetty-util-6.1.26.j
ar:/root/apache-storm-0.9.2-incubating/lib/objenesis-1.2.jar:/r
oot/apache-storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/root
/apache-storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.ja
r:/root/apache-storm-0.9.2-incubating/lib/commons-codec-1.6.jar
:/root/apache-storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/roo
t/apache-storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/root
/apache-storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar
:/root/apache-storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/r
oot/apache-storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/roo
t/apache-storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/root/
apache-storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/root/a
pache-storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/root/apache-
storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/root/apache-stor
m-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/root/apach
e-storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/root/apac
he-storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/root/SOIM
/target/SOIMTopology-1.0-SNAPSHOT.jar:/root/.storm:/root/apache
-storm-0.9.2-incubating/bin
-Dstorm.jar=/root/SOIM/target/SOIMTopology-1.0-SNAPSHOT.jar
Microland.SOIM.SOIMTopology SOIMTopology
404 [main] INFO backtype.storm.StormSubmitter - Uploading
topology jar /root/SOIM/target/SOIMTopology-1.0-SNAPSHOT.jar to
assigned location:
storm-local/nimbus/inbox/stormjar-b867da73-26e6-474d-9e80-3d469
0adbfa3.jar
421 [main] INFO backtype.storm.StormSubmitter - Successfully
uploaded topology jar to assigned location:
storm-local/nimbus/inbox/stormjar-b867da73-26e6-474d-9e80-3d469
0adbfa3.jar
Regards,
PaVan…
From: Supun Kamburugamuva [[12]mailto:[email protected]]
Sent: 01 September 2014 20:04
To: [13][email protected]
Subject: Re: REMOTE MODE STORM DEV
Do you get any errors in the console when deploying the
topology? You can also check supervisor logs for errors.
Thanks,
Supun..
On Mon, Sep 1, 2014 at 8:03 AM, Pavan Jakati G
<[14][email protected]> wrote:
Hi Team ,
We have create a topology which run fine in local mode but fail
to action when in remote mode . Do we need make any changes to
Spout and Bolt to make it run in remote mode ?. Our code has
similar lines as shown here in these links . Guidance would be
appreciated . Thanks.
REFERENCE LINK:
[15]http://mail-archives.apache.org/mod_mbox/storm-user/201404.
mbox/%3CCAJq5PB5pDjYemyf3m+rVyjPzN3_t9_-d8nF-7rWEJVhNXB=Yag@mai
l.gmail.com%3E
MAIN FUNCTION
[16]http://mail-archives.apache.org/mod_mbox/storm-user/201404.
mbox/raw/%3CCAJq5PB5pDjYemyf3m+rVyjPzN3_t9_-d8nF-7rWEJVhNXB=Yag
@mail.gmail.com%3E/2
SPOUT
[17]http://mail-archives.apache.org/mod_mbox/storm-user/201404.
mbox/raw/%3CCAJq5PB5pDjYemyf3m+rVyjPzN3_t9_-d8nF-7rWEJVhNXB=Yag
@mail.gmail.com%3E/3
BOLT
[18]http://mail-archives.apache.org/mod_mbox/storm-user/201404.
mbox/raw/%3CCAJq5PB5pDjYemyf3m+rVyjPzN3_t9_-d8nF-7rWEJVhNXB=Yag
@mail.gmail.com%3E/4
Regards
PaVan ..
Stay Hungry . Stay Foolish J
The information contained in this transmission may contain
privileged and confidential information of Microland Limited,
including information protected by privacy laws. It is intended
only for the use of Microland Limited. If you are not the
intended recipient, you are hereby notified that any review,
dissemination, distribution, or duplication of this
communication is strictly prohibited. If you are not the
intended recipient, please contact the sender by reply email
and destroy all copies of the original message.
--
Supun Kamburugamuva
Member, Apache Software Foundation; [19]http://www.apache.org
E-mail: [20][email protected]; Mobile: [21]+1 812 369 6762
Blog: [22]http://supunk.blogspot.com
--
Regards,
Vikas Agarwal
91 – 9928301411
InfoObjects, Inc.
Execution Matters
[23]http://www.infoobjects.com
2041 Mission College Boulevard, #280
Santa Clara, CA 95054
[24]+1 (408) 988-2000 Work
[25]+1 (408) 716-2726 Fax
--
Supun Kamburugamuva
Member, Apache Software Foundation; [26]http://www.apache.org
E-mail: [27][email protected]; Mobile: +1 812 369 6762
Blog: [28]http://supunk.blogspot.com
References
1. mailto:[email protected]
2. mailto:[email protected]
3. mailto:[email protected]
4. mailto:[email protected]
5. mailto:[email protected]
6. http://storm.id/
7. http://storm.messaging.netty.flush.check.interval.ms/
8. http://topology.name/
9. http://topology.sleep.spout.wait.strategy.time.ms/
10. http://logviewer.appender.name/
11. mailto:[email protected]
12. mailto:[email protected]
13. mailto:[email protected]
14. mailto:[email protected]
15.
http://mail-archives.apache.org/mod_mbox/storm-user/201404.mbox/%3CCAJq5PB5pDjYemyf3m+rVyjPzN3_t9_-d8nF-7rWEJVhNXB=y...@mail.gmail.com%3E
16.
http://mail-archives.apache.org/mod_mbox/storm-user/201404.mbox/raw/%3CCAJq5PB5pDjYemyf3m+rVyjPzN3_t9_-d8nF-7rWEJVhNXB=y...@mail.gmail.com%3E/2
17.
http://mail-archives.apache.org/mod_mbox/storm-user/201404.mbox/raw/%3CCAJq5PB5pDjYemyf3m+rVyjPzN3_t9_-d8nF-7rWEJVhNXB=y...@mail.gmail.com%3E/3
18.
http://mail-archives.apache.org/mod_mbox/storm-user/201404.mbox/raw/%3CCAJq5PB5pDjYemyf3m+rVyjPzN3_t9_-d8nF-7rWEJVhNXB=y...@mail.gmail.com%3E/4
19. http://www.apache.org/
20. mailto:[email protected]
21. tel:%2B1%20812%20369%206762
22. http://supunk.blogspot.com/
23. http://www.infoobjects.com/
24. tel:%2B1%20%28408%29%20988-2000
25. tel:%2B1%20%28408%29%20716-2726
26. http://www.apache.org/
27. mailto:[email protected]
28. http://supunk.blogspot.com/
