Pavan,
Which user starting the storm daemons, from your previous
emails its looks like you are starting them as user "storm" .
storm dir is owned by root and storm daemons tries to write to
storm-local and also log files which might be what causing
issues. I recommend you to go through this
tutorial [1]http://www.michael-noll.com/tutorials/running-multi
-node-storm-cluster/
Although its a multinode cluster setup you can adapt it to
single host. Running storm or such services as root is bad
idea.
Above tutorial talks about creating user storm and running the
services as "storm" user. Try to setup your installation that
way.
-Harsha
On Wed, Sep 3, 2014, at 12:09 AM, Pavan Jakati G wrote:
Hi Harsha,
I am running it on single host . Attached is the storm.yaml
file . Permissions of the directory is as below ,
ls -ld /root/apache-storm-0.9.2-incubating
drwxrwxrwx 11 root root 4096 Sep 2 05:33
/root/apache-storm-0.9.2-incubating
ll /root/apache-storm-0.9.2-incubating
total 120
drwxrwxrwx 3 root root 4096 Sep 1 09:49 bin
-rw-r--r-- 1 root root 34239 Jun 12 20:46 CHANGELOG.md
drwxrwxrwx 2 root root 4096 Sep 1 12:31 conf
-rw-r--r-- 1 root root 538 Mar 12 23:17 DISCLAIMER
drwxrwxrwx 3 root root 4096 Jun 16 12:22 examples
drwxrwxrwx 3 root root 4096 Jun 16 12:22 external
drwxrwxrwx 2 root root 4096 Jun 16 12:22 lib
-rw-r--r-- 1 root root 22822 Jun 11 16:07 LICENSE
drwxrwxrwx 2 root root 4096 Jun 16 12:22 logback
drwxr-xr-x 3 root root 4096 Sep 2 13:08 logs
-rw-r--r-- 1 root root 981 Jun 10 13:10 NOTICE
drwxrwxrwx 5 root root 4096 Jun 16 12:22 public
-rw-r--r-- 1 root root 7445 Jun 9 14:24 README.markdown
-rw-r--r-- 1 root root 17 Jun 16 12:22 RELEASE
-rw-r--r-- 1 root root 3581 May 29 12:20 SECURITY.md
drwxr-xr-x 4 root root 4096 Sep 2 05:34 storm-local
Regards,
PaVan…
From: Harsha [mailto:[email protected]]
Sent: 03 September 2014 00:02
To: [email protected]
Subject: Re: REMOTE MODE STORM DEV
Pavan,
It would be helpful if you can post your storm.yaml.
Make sure user "storm" has permissions to your storm
installation dir. and you used the same storm config on all
your machines.
-Harsha
On Tue, Sep 2, 2014, at 06:53 AM, Supun Kamburugamuva wrote:
Hi Pavan,
It seems you have a permission issue. Please check weather the
directories that the storm user has appropriate permissions on
the directories that contain the storm jars.
Thanks,
Supun..
On Tue, Sep 2, 2014 at 9:13 AM, Pavan Jakati G
<[2][email protected]> wrote:
Can anybody help us get rid of below error :
sudo -u storm /usr/java/jdk1.7.0_65/bin/java -server -Xmx768m
-Djava.library.path=storm-local/supervisor/stormdist/PaVan-14-1
409661199/resources/Linux-amd64:storm-local/supervisor/stormdis
t/PaVan-14-1409661199/resources:/usr/local/lib:/opt/local/lib:/
usr/lib -Dlogfile.name=worker-6703.log
-Dstorm.home=/root/apache-storm-0.9.2-incubating
-Dlogback.configurationFile=/root/apache-storm-0.9.2-incubating
/logback/cluster.xml -Dstorm.id=PaVan-14-1409661199
-Dworker.id=156a8af9-fa3b-4772-b91c-787490fe0b34
-Dworker.port=6703 -cp
/root/apache-storm-0.9.2-incubating/lib/json-simple-1.1.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/
root/apache-storm-0.9.2-incubating/lib/asm-4.0.jar:/root/apache
-storm-0.9.2-incubating/lib/commons-io-2.4.jar:/root/apache-sto
rm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/root/apache-storm
-0.9.2-incubating/lib/jline-2.11.jar:/root/apache-storm-0.9.2-i
ncubating/lib/tools.logging-0.2.3.jar:/root/apache-storm-0.9.2-
incubating/lib/logback-classic-1.0.6.jar:/root/apache-storm-0.9
.2-incubating/lib/commons-logging-1.1.3.jar:/root/apache-storm-
0.9.2-incubating/lib/ring-core-1.1.5.jar:/root/apache-storm-0.9
.2-incubating/lib/ring-devel-0.3.11.jar:/root/apache-storm-0.9.
2-incubating/lib/curator-client-2.4.0.jar:/root/apache-storm-0.
9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/root/apache-storm-
0.9.2-incubating/lib/clj-time-0.4.1.jar:/root/apache-storm-0.9.
2-incubating/lib/commons-lang-2.5.jar:/root/apache-storm-0.9.2-
incubating/lib/zookeeper-3.4.5.jar:/root/apache-storm-0.9.2-inc
ubating/lib/compojure-1.1.3.jar:/root/apache-storm-0.9.2-incuba
ting/lib/joda-time-2.0.jar:/root/apache-storm-0.9.2-incubating/
lib/chill-java-0.3.5.jar:/root/apache-storm-0.9.2-incubating/li
b/clout-1.0.1.jar:/root/apache-storm-0.9.2-incubating/lib/kryo-
2.21.jar:/root/apache-storm-0.9.2-incubating/lib/snakeyaml-1.11
.jar:/root/apache-storm-0.9.2-incubating/lib/minlog-1.2.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/storm-core-0.9.2-incubatin
g.jar:/root/apache-storm-0.9.2-incubating/lib/jgrapht-core-0.9.
0.jar:/root/apache-storm-0.9.2-incubating/lib/slf4j-api-1.6.5.j
ar:/root/apache-storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/roo
t/apache-storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar
:/root/apache-storm-0.9.2-incubating/lib/guava-13.0.jar:/root/a
pache-storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.j
ar:/root/apache-storm-0.9.2-incubating/lib/servlet-api-2.5.jar:
/root/apache-storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.
jar:/root/apache-storm-0.9.2-incubating/lib/jetty-util-6.1.26.j
ar:/root/apache-storm-0.9.2-incubating/lib/objenesis-1.2.jar:/r
oot/apache-storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/root
/apache-storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.ja
r:/root/apache-storm-0.9.2-incubating/lib/commons-codec-1.6.jar
:/root/apache-storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/roo
t/apache-storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/root
/apache-storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar
:/root/apache-storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/r
oot/apache-storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/roo
t/apache-storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/root/
apache-storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/root/a
pache-storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/root/apache-
storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/root/apache-stor
m-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/root/apach
e-storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/root/apac
he-storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/root/apac
he-storm-0.9.2-incubating/conf:storm-local/supervisor/stormdist
/PaVan-14-1409661199/stormjar.jar backtype.storm.daemon.worker
PaVan-14-1409661199 0574446a-f73d-42b0-bcc7-e6dd449cb75a 6703
156a8af9-fa3b-4772-b91c-787490fe0b34
Error: Could not find or load main class
backtype.storm.daemon.worker
Regards,
PaVan…
From: Pavan Jakati G
Sent: 02 September 2014 15:23
To: [3][email protected]
Subject: RE: REMOTE MODE STORM DEV
sudo -u storm '/usr/java/jdk1.7.0_65/bin/java' '-server'
'-Xmx768m' '-Djava.library.path=storm-local/sup
ervisor/stormdist/PaVan-10-1409648607/resources/Linux-amd64:sto
rm-local/supervisor/stormdist/PaVan-10-1409648607/resources:/us
r/local/lib:/opt/local/lib:/usr/lib'
'-Dlogfile.name=worker-6701.log'
'-Dstorm.home=/root/apache-storm-0.9.2-incubating'
'-Dlogback.configurationFile=/root/apache-storm-0.9.2-incubatin
g/logback/cluster.xml' '-Dstorm.id=PaVan-10-1409648607'
'-Dworker.id=b89ee630-7bd6-4976-8050-c43644c028ed'
'-Dworker.port=6701' '-cp'
'/root/apache-storm-0.9.2-incubating/lib/json-simple-1.1.jar:/r
oot/apache-storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:
/root/apache-storm-0.9.2-incubating/lib/asm-4.0.jar:/root/apach
e-storm-0.9.2-incubating/lib/commons-io-2.4.jar:/root/apache-st
orm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/root/apache-stor
m-0.9.2-incubating/lib/jline-2.11.jar:/root/apache-storm-0.9.2-
incubating/lib/tools.logging-0.2.3.jar:/root/apache-storm-0.9.2
-incubating/lib/logback-classic-1.0.6.jar:/root/apache-storm-0.
9.2-incubating/lib/commons-logging-1.1.3.jar:/root/apache-storm
-0.9.2-incubating/lib/ring-core-1.1.5.jar:/root/apache-storm-0.
9.2-incubating/lib/ring-devel-0.3.11.jar:/root/apache-storm-0.9
.2-incubating/lib/curator-client-2.4.0.jar:/root/apache-storm-0
.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/root/apache-storm
-0.9.2-incubating/lib/clj-time-0.4.1.jar:/root/apache-storm-0.9
.2-incubating/lib/commons-lang-2.5.jar:/root/apache-storm-0.9.2
-incubating/lib/zookeeper-3.4.5.jar:/root/apache-storm-0.9.2-in
cubating/lib/compojure-1.1.3.jar:/root/apache-storm-0.9.2-incub
ating/lib/joda-time-2.0.jar:/root/apache-storm-0.9.2-incubating
/lib/chill-java-0.3.5.jar:/root/apache-storm-0.9.2-incubating/l
ib/clout-1.0.1.jar:/root/apache-storm-0.9.2-incubating/lib/kryo
-2.21.jar:/root/apache-storm-0.9.2-incubating/lib/snakeyaml-1.1
1.jar:/root/apache-storm-0.9.2-incubating/lib/minlog-1.2.jar:/r
oot/apache-storm-0.9.2-incubating/lib/storm-core-0.9.2-incubati
ng.jar:/root/apache-storm-0.9.2-incubating/lib/jgrapht-core-0.9
.0.jar:/root/apache-storm-0.9.2-incubating/lib/slf4j-api-1.6.5.
jar:/root/apache-storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/r
oot/apache-storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/curator-framework-2.4.0.ja
r:/root/apache-storm-0.9.2-incubating/lib/guava-13.0.jar:/root/
apache-storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/r
oot/apache-storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.
jar:/root/apache-storm-0.9.2-incubating/lib/servlet-api-2.5.jar
:/root/apache-storm-0.9.2-incubating/lib/reflectasm-1.07-shaded
.jar:/root/apache-storm-0.9.2-incubating/lib/jetty-util-6.1.26.
jar:/root/apache-storm-0.9.2-incubating/lib/objenesis-1.2.jar:/
root/apache-storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/roo
t/apache-storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.j
ar:/root/apache-storm-0.9.2-incubating/lib/commons-codec-1.6.ja
r:/root/apache-storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/roo
t/apache-storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.ja
r:/root/apache-storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/
root/apache-storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/root
/apache-storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/root/
apache-storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/root/apache
-storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/root/apache-sto
rm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/root/apac
he-storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/root/apa
che-storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/root/apa
che-storm-0.9.2-incubating/conf:storm-local/supervisor/stormdis
t/PaVan-10-1409648607/stormjar.jar'
'backtype.storm.daemon.worker' 'PaVan-10-1409648607'
'0574446a-f73d-42b0-bcc7-e6dd449cb75a' '6701'
'b89ee630-7bd6-4976-8050-c43644c028ed'
Error: Could not find or load main class
backtype.storm.daemon.worker
Regards,
PaVan…
From: Vikas Agarwal [[4]mailto:[email protected]]
Sent: 02 September 2014 15:05
To: [5][email protected]
Subject: Re: REMOTE MODE STORM DEV
Pavan,
Have you tried running the command for launching worker as
logged in supervisor.log? You need to run the command with
storm user and may need to kill existing one if running on same
port.
On Tue, Sep 2, 2014 at 3:02 PM, Pavan Jakati G
<[6][email protected]> wrote:
Is supervisor running or dead ? . Spouts and bolts which run
fine in local mode fails to do same in remote mode. Pls help us
here.
less supervisor.log
m-0.9.2-incubating/lib/commons-io-2.4.jar:/root/apache-storm-0.
9.2-incubating/lib/httpclient-4.3.3.jar:/root/apache-storm-0.9.
2-incubating/lib/jline-2.11.jar:/root/apache-storm-0.9.2-incuba
ting/lib/tools.logging-0.2.3.jar:/root/apache-storm-0.9.2-incub
ating/lib/logback-classic-1.0.6.jar:/root/apache-storm-0.9.2-in
cubating/lib/commons-logging-1.1.3.jar:/root/apache-storm-0.9.2
-incubating/lib/ring-core-1.1.5.jar:/root/apache-storm-0.9.2-in
cubating/lib/ring-devel-0.3.11.jar:/root/apache-storm-0.9.2-inc
ubating/lib/curator-client-2.4.0.jar:/root/apache-storm-0.9.2-i
ncubating/lib/clj-stacktrace-0.2.4.jar:/root/apache-storm-0.9.2
-incubating/lib/clj-time-0.4.1.jar:/root/apache-storm-0.9.2-inc
ubating/lib/commons-lang-2.5.jar:/root/apache-storm-0.9.2-incub
ating/lib/zookeeper-3.4.5.jar:/root/apache-storm-0.9.2-incubati
ng/lib/compojure-1.1.3.jar:/root/apache-storm-0.9.2-incubating/
lib/joda-time-2.0.jar:/root/apache-storm-0.9.2-incubating/lib/c
hill-java-0.3.5.jar:/root/apache-storm-0.9.2-incubating/lib/clo
ut-1.0.1.jar:/root/apache-storm-0.9.2-incubating/lib/kryo-2.21.
jar:/root/apache-storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:
/root/apache-storm-0.9.2-incubating/lib/minlog-1.2.jar:/root/ap
ache-storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar
:/root/apache-storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar
:/root/apache-storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/r
oot/apache-storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/root/ap
ache-storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/root/apa
che-storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/roo
t/apache-storm-0.9.2-incubating/lib/guava-13.0.jar:/root/apache
-storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/root/ap
ache-storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/r
oot/apache-storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/root
/apache-storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/
root/apache-storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/r
oot/apache-storm-0.9.2-incubating/lib/objenesis-1.2.jar:/root/a
pache-storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/root/apac
he-storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/roo
t/apache-storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/root/apa
che-storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/root/apac
he-storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/roo
t/apache-storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/root/a
pache-storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/root/apa
che-storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/root/apach
e-storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/root/apache
-storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/root/apache-storm
-0.9.2-incubating/lib/httpcore-4.3.2.jar:/root/apache-storm-0.9
.2-incubating/lib/servlet-api-2.5-20081211.jar:/root/apache-sto
rm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/root/apache-st
orm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/root/apache-st
orm-0.9.2-incubating/conf:storm-local/supervisor/stormdist/PRIM
ETopology-6-1409635517/stormjar.jar'
'backtype.storm.daemon.worker' 'PRIMETopology-6-1409635517'
'0574446a-f73d-42b0-bcc7-e6dd449cb75a' '6702'
'ddf2747a-276a-4724-ba08-4f5d46b0b5ec'
2014-09-02 05:34:01 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:02 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:02 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:03 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:03 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:04 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:04 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:05 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:05 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:06 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:06 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:07 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:07 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:08 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:08 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
2014-09-02 05:34:09 b.s.d.supervisor [INFO]
5a5f5d59-d67d-4301-b899-36e5432212e1 still hasn't started
Worker Log:
2014-09-02 09:03:37 b.s.d.executor [INFO] Loading executor
SOIMbolt:[1 1]
2014-09-02 09:03:37 b.s.d.executor [INFO] Loaded executor tasks
SOIMbolt:[1 1]
2014-09-02 09:03:37 b.s.d.executor [INFO] Prepared bolt
__system:(-1)
2014-09-02 09:03:37 b.s.d.executor [INFO] Finished loading
executor SOIMbolt:[1 1]
2014-09-02 09:03:37 b.s.d.worker [INFO] Launching
receive-thread for 0574446a-f73d-42b0-bcc7-e6dd449cb75a:6701
2014-09-02 09:03:37 b.s.d.executor [INFO] Preparing bolt
SOIMbolt:(1)
2014-09-02 09:03:37 b.s.d.executor [INFO] Prepared bolt
SOIMbolt:(1)
2014-09-02 09:03:37 b.s.m.n.Server [INFO] Create Netty Server
Netty-server-localhost-6701, buffer_size: 5242880, maxWorkers:
1
2014-09-02 09:03:38 b.s.m.loader [INFO] Starting
receive-thread: [stormId: PaVan-10-1409648607, port: 6701,
thread-id: 0 ]
2014-09-02 09:03:38 b.s.d.worker [INFO] Worker has topology
config {"[7]storm.id" "PaVan-10-1409648607",
"dev.zookeeper.path" "/tmp/dev-storm-zookeeper",
"topology.tick.tuple.freq.secs" nil,
"topology.builtin.metrics.bucket.size.secs" 60,
"topology.fall.back.on.java.serialization" true,
"topology.max.error.report.per.interval" 5, "zmq.linger.millis"
5000, "topology.skip.missing.kryo.registrations" false,
"storm.messaging.netty.client_worker_threads" 1, "ui.childopts"
"-Xmx768m", "storm.zookeeper.session.timeout" 20000,
"nimbus.reassign" true,
"topology.trident.batch.emit.interval.millis" 500,
"drpc.servers" ["192.168.10.7"],
"[8]storm.messaging.netty.flush.check.interval.ms" 10,
"nimbus.monitor.freq.secs" 10, "logviewer.childopts"
"-Xmx128m", "java.library.path"
"/usr/local/lib:/opt/local/lib:/usr/lib",
"topology.executor.send.buffer.size" 1024, "storm.local.dir"
"storm-local", "storm.messaging.netty.buffer_size" 5242880,
"supervisor.worker.start.timeout.secs" 120,
"topology.enable.message.timeouts" true,
"nimbus.cleanup.inbox.freq.secs" 600,
"nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads"
64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host"
"192.168.10.7", "storm.messaging.netty.min_wait_ms" 100,
"storm.zookeeper.port" 2181, "transactional.zookeeper.port"
nil, "topology.executor.receive.buffer.size" 1024,
"transactional.zookeeper.servers" nil, "storm.zookeeper.root"
"/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000,
"supervisor.enable" true,
"storm.messaging.netty.server_worker_threads" 1,
"storm.zookeeper.servers" ["localhost"],
"transactional.zookeeper.root" "/transactional",
"topology.acker.executors" nil, "topology.kryo.decorators" (),
"[9]topology.name" "PaVan", "topology.transfer.buffer.size"
1024, "topology.worker.childopts" nil, "drpc.queue.size" 128,
"worker.childopts" "-Xmx768m",
"supervisor.heartbeat.frequency.secs" 5,
"topology.error.throttle.interval.secs" 10, "zmq.hwm" 0,
"drpc.port" 3772, "supervisor.monitor.frequency.secs" 3,
"drpc.childopts" "-Xmx768m", "topology.receiver.buffer.size" 8,
"task.heartbeat.frequency.secs" 3, "topology.tasks" nil,
"storm.messaging.netty.max_retries" 30,
"topology.spout.wait.strategy"
"backtype.storm.spout.SleepSpoutWaitStrategy",
"nimbus.thrift.max_buffer_size" 1048576,
"topology.max.spout.pending" nil,
"storm.zookeeper.retry.interval" 1000,
"[10]topology.sleep.spout.wait.strategy.time.ms" 1,
"nimbus.topology.validator"
"backtype.storm.nimbus.DefaultTopologyValidator",
"supervisor.slots.ports" [6700 6701 6702 6703],
"topology.debug" false, "nimbus.task.launch.secs" 120,
"nimbus.supervisor.timeout.secs" 60, "topology.kryo.register"
nil, "topology.message.timeout.secs" 30,
"task.refresh.poll.secs" 10, "topology.workers" 1,
"supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627,
"topology.stats.sample.rate" 0.05,
"worker.heartbeat.frequency.secs" 1,
"topology.tuple.serializer"
"backtype.storm.serialization.types.ListDelegateSerializer",
"topology.disruptor.wait.strategy"
"com.lmax.disruptor.BlockingWaitStrategy",
"topology.multilang.serializer"
"backtype.storm.multilang.JsonSerializer",
"nimbus.task.timeout.secs" 30,
"storm.zookeeper.connection.timeout" 15000,
"topology.kryo.factory"
"backtype.storm.serialization.DefaultKryoFactory",
"drpc.invocations.port" 3773, "logviewer.port" 8000,
"zmq.threads" 1, "storm.zookeeper.retry.times" 5,
"topology.worker.receiver.thread.count" 1,
"storm.thrift.transport"
"backtype.storm.security.auth.SimpleTransportPlugin",
"topology.state.synchronization.timeout.secs" 60,
"supervisor.worker.timeout.secs" 30,
"nimbus.file.copy.expiration.secs" 600,
"storm.messaging.transport"
"backtype.storm.messaging.netty.Context",
"[11]logviewer.appender.name" "A1",
"storm.messaging.netty.max_wait_ms" 1000,
"drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false,
"ui.port" 8080, "nimbus.childopts" "-Xmx1024m",
"storm.cluster.mode" "distributed",
"topology.max.task.parallelism" nil,
"storm.messaging.netty.transfer.batch.size" 262144}
2014-09-02 09:03:38 b.s.d.worker [INFO] Worker
b89ee630-7bd6-4976-8050-c43644c028ed for storm
PaVan-10-1409648607 on
0574446a-f73d-42b0-bcc7-e6dd449cb75a:6701 has finished loading
Spout Code :
public class SOIMSpout extends BaseRichSpout
{
private SpoutOutputCollector collector;
boolean _isDistributed;
java.sql.ResultSet SOIMResultset = null;
java.sql.Connection SOIMConnection = null;
java.sql.Statement SOIMStatement = null;
private PreparedStatement SOIMPrepstatement = null;
private PreparedStatement SOIMPrepstatement_Stream =
null;
private PreparedStatement SOIMPrepstatementTotal =
null;
int SOIMTicketno = 0 ;
String SOIMDesc = "", SOIMInf = "", SOIMDate = "",
SOIMDevice = "", SOIMCategory = "", SOIMSubCategory = "",
SOIMPriority="";
String SOIMdbUrl = "", SOIMdbUser = "", SOIMdbPassword
= "", SOIMdbClass="";
/************ MYSQL DB CONNECTION VARIABLES FOR
RECEIVING ************/
public SOIMSpout()
{
SOIMConnector objSOIMConnect = new SOIMConnector();
/********* OBJECT CREATION FOR SOIMConnector CLASS -
SRIDHAR ********/
SOIMdbUrl = objSOIMConnect.dbUrl;
SOIMdbUser = objSOIMConnect.dbUser;
SOIMdbPassword = objSOIMConnect.dbPassword;
}
@Override
public void nextTuple()
{
try
{
SOIMConnection =
DriverManager.getConnection(SOIMdbUrl,SOIMdbUser,SOIMdbPassword
);
//******* Object for jdbc connectivity from
SOIMConnector class -- Sridhar*** //
SOIMStatement =
SOIMConnection.createStatement();
String SOIMQuery = "SELECT
ticketno,description,information,createdon,configitemname,categ
oryname,technologyname,priorityname FROM siom_tosticket";
SOIMResultset =
SOIMStatement.executeQuery(SOIMQuery);
while(SOIMResultset.next())
{
SOIMTicketno =
(SOIMResultset.getInt("ticketno"));
SOIMDesc =
(SOIMResultset.getString("description"));
SOIMInf=
(SOIMResultset.getString("information"));
SOIMDate =
(SOIMResultset.getString("createdon"));
SOIMDevice =
(SOIMResultset.getString("configitemname"));
SOIMCategory =
(SOIMResultset.getString("categoryname"));
SOIMSubCategory =
(SOIMResultset.getString("technologyname"));
SOIMPriority=(SOIMResultset.getString("priorityname"));
//collector.emit(new
Values(SOIMTicketno,SOIMDesc,
SOIMInf,SOIMDate,SOIMDevice,SOIMCategory,SOIMSubCategory,SOIMPr
iority));
//******* Emitting Tuples from
SPOUT to BOLT -- Sridhar*** //
}
collector.emit(new Values(SOIMTicketno,SOIMDesc,
SOIMInf,SOIMDate,SOIMDevice,SOIMCategory,SOIMSubCategory,SOIMPr
iority));
}
catch (Exception e) {
}
finally {
if (SOIMConnection != null) {
@Override
public void ack(Object id)
{
}
@Override
public void fail(Object id)
{
}
@Override
public void declareOutputFields(OutputFieldsDeclarer
declarer)
{
declarer.declare( new Fields("SOIMTicketno","SOIMDesc",
"SOIMInf","SOIMDate","SOIMDevice","SOIMCategory","SOIMSubCatego
ry","SOIMPriority"));
}
@Override
public void close() {
}
@Override
public void activate() {
// TODO Auto-generated method stub
}
@Override
public void deactivate() {
// TODO Auto-generated method stub
}
Regards,
PaVan…
From: Pavan Jakati G
Sent: 02 September 2014 10:56
To: [12][email protected]
Subject: RE: REMOTE MODE STORM DEV
storm jar /root/SOIM/target/SOIMTopology-1.0-SNAPSHOT.jar
Microland.SOIM.SOIMTopology SOIMTopology
Running: /usr/java/jdk1.7.0_65/bin/java -client
-Dstorm.options=
-Dstorm.home=/root/apache-storm-0.9.2-incubating
-Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
-Dstorm.conf.file= -cp
/root/apache-storm-0.9.2-incubating/lib/json-simple-1.1.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/
root/apache-storm-0.9.2-incubating/lib/asm-4.0.jar:/root/apache
-storm-0.9.2-incubating/lib/commons-io-2.4.jar:/root/apache-sto
rm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/root/apache-storm
-0.9.2-incubating/lib/jline-2.11.jar:/root/apache-storm-0.9.2-i
ncubating/lib/tools.logging-0.2.3.jar:/root/apache-storm-0.9.2-
incubating/lib/logback-classic-1.0.6.jar:/root/apache-storm-0.9
.2-incubating/lib/commons-logging-1.1.3.jar:/root/apache-storm-
0.9.2-incubating/lib/ring-core-1.1.5.jar:/root/apache-storm-0.9
.2-incubating/lib/ring-devel-0.3.11.jar:/root/apache-storm-0.9.
2-incubating/lib/curator-client-2.4.0.jar:/root/apache-storm-0.
9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/root/apache-storm-
0.9.2-incubating/lib/clj-time-0.4.1.jar:/root/apache-storm-0.9.
2-incubating/lib/commons-lang-2.5.jar:/root/apache-storm-0.9.2-
incubating/lib/zookeeper-3.4.5.jar:/root/apache-storm-0.9.2-inc
ubating/lib/compojure-1.1.3.jar:/root/apache-storm-0.9.2-incuba
ting/lib/joda-time-2.0.jar:/root/apache-storm-0.9.2-incubating/
lib/chill-java-0.3.5.jar:/root/apache-storm-0.9.2-incubating/li
b/clout-1.0.1.jar:/root/apache-storm-0.9.2-incubating/lib/kryo-
2.21.jar:/root/apache-storm-0.9.2-incubating/lib/snakeyaml-1.11
.jar:/root/apache-storm-0.9.2-incubating/lib/minlog-1.2.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/storm-core-0.9.2-incubatin
g.jar:/root/apache-storm-0.9.2-incubating/lib/jgrapht-core-0.9.
0.jar:/root/apache-storm-0.9.2-incubating/lib/slf4j-api-1.6.5.j
ar:/root/apache-storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/roo
t/apache-storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar
:/root/apache-storm-0.9.2-incubating/lib/guava-13.0.jar:/root/a
pache-storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/ro
ot/apache-storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.j
ar:/root/apache-storm-0.9.2-incubating/lib/servlet-api-2.5.jar:
/root/apache-storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.
jar:/root/apache-storm-0.9.2-incubating/lib/jetty-util-6.1.26.j
ar:/root/apache-storm-0.9.2-incubating/lib/objenesis-1.2.jar:/r
oot/apache-storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/root
/apache-storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.ja
r:/root/apache-storm-0.9.2-incubating/lib/commons-codec-1.6.jar
:/root/apache-storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/roo
t/apache-storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/root
/apache-storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar
:/root/apache-storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/r
oot/apache-storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/roo
t/apache-storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/root/
apache-storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/root/a
pache-storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/root/apache-
storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/root/apache-stor
m-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/root/apach
e-storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/root/apac
he-storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/root/SOIM
/target/SOIMTopology-1.0-SNAPSHOT.jar:/root/.storm:/root/apache
-storm-0.9.2-incubating/bin
-Dstorm.jar=/root/SOIM/target/SOIMTopology-1.0-SNAPSHOT.jar
Microland.SOIM.SOIMTopology SOIMTopology
404 [main] INFO backtype.storm.StormSubmitter - Uploading
topology jar /root/SOIM/target/SOIMTopology-1.0-SNAPSHOT.jar to
assigned location:
storm-local/nimbus/inbox/stormjar-b867da73-26e6-474d-9e80-3d469
0adbfa3.jar
421 [main] INFO backtype.storm.StormSubmitter - Successfully
uploaded topology jar to assigned location:
storm-local/nimbus/inbox/stormjar-b867da73-26e6-474d-9e80-3d469
0adbfa3.jar
Regards,
PaVan…
From: Supun Kamburugamuva [[13]mailto:[email protected]]
Sent: 01 September 2014 20:04
To: [14][email protected]
Subject: Re: REMOTE MODE STORM DEV
Do you get any errors in the console when deploying the
topology? You can also check supervisor logs for errors.
Thanks,
Supun..
On Mon, Sep 1, 2014 at 8:03 AM, Pavan Jakati G
<[15][email protected]> wrote:
Hi Team ,
We have create a topology which run fine in local mode but fail
to action when in remote mode . Do we need make any changes to
Spout and Bolt to make it run in remote mode ?. Our code has
similar lines as shown here in these links . Guidance would be
appreciated . Thanks.
REFERENCE LINK:
[16]http://mail-archives.apache.org/mod_mbox/storm-user/201404.
mbox/%3CCAJq5PB5pDjYemyf3m+rVyjPzN3_t9_-d8nF-7rWEJVhNXB=Yag@mai
l.gmail.com%3E
MAIN FUNCTION
[17]http://mail-archives.apache.org/mod_mbox/storm-user/201404.
mbox/raw/%3CCAJq5PB5pDjYemyf3m+rVyjPzN3_t9_-d8nF-7rWEJVhNXB=Yag
@mail.gmail.com%3E/2
SPOUT
[18]http://mail-archives.apache.org/mod_mbox/storm-user/201404.
mbox/raw/%3CCAJq5PB5pDjYemyf3m+rVyjPzN3_t9_-d8nF-7rWEJVhNXB=Yag
@mail.gmail.com%3E/3
BOLT
[19]http://mail-archives.apache.org/mod_mbox/storm-user/201404.
mbox/raw/%3CCAJq5PB5pDjYemyf3m+rVyjPzN3_t9_-d8nF-7rWEJVhNXB=Yag
@mail.gmail.com%3E/4
Regards
PaVan ..
Stay Hungry . Stay Foolish J
The information contained in this transmission may contain
privileged and confidential information of Microland Limited,
including information protected by privacy laws. It is intended
only for the use of Microland Limited. If you are not the
intended recipient, you are hereby notified that any review,
dissemination, distribution, or duplication of this
communication is strictly prohibited. If you are not the
intended recipient, please contact the sender by reply email
and destroy all copies of the original message.
--
Supun Kamburugamuva
Member, Apache Software Foundation; [20]http://www.apache.org
E-mail: [21][email protected]; Mobile: [22]+1 812 369 6762
Blog: [23]http://supunk.blogspot.com
--
Regards,
Vikas Agarwal
91 – 9928301411
InfoObjects, Inc.
Execution Matters
[24]http://www.infoobjects.com
2041 Mission College Boulevard, #280
Santa Clara, CA 95054
[25]+1 (408) 988-2000 Work
[26]+1 (408) 716-2726 Fax
--
Supun Kamburugamuva
Member, Apache Software Foundation; [27]http://www.apache.org
E-mail: [28][email protected]; Mobile: +1 812 369 6762
Blog: [29]http://supunk.blogspot.com
Email had 1 attachment:
* storm.yaml
2k (application/octet-stream)
References
1. http://www.michael-noll.com/tutorials/running-multi-node-storm-cluster/
2. mailto:[email protected]
3. mailto:[email protected]
4. mailto:[email protected]
5. mailto:[email protected]
6. mailto:[email protected]
7. http://storm.id/
8. http://storm.messaging.netty.flush.check.interval.ms/
9. http://topology.name/
10. http://topology.sleep.spout.wait.strategy.time.ms/
11. http://logviewer.appender.name/
12. mailto:[email protected]
13. mailto:[email protected]
14. mailto:[email protected]
15. mailto:[email protected]
16.
http://mail-archives.apache.org/mod_mbox/storm-user/201404.mbox/%3CCAJq5PB5pDjYemyf3m+rVyjPzN3_t9_-d8nF-7rWEJVhNXB=y...@mail.gmail.com%3E
17.
http://mail-archives.apache.org/mod_mbox/storm-user/201404.mbox/raw/%3CCAJq5PB5pDjYemyf3m+rVyjPzN3_t9_-d8nF-7rWEJVhNXB=y...@mail.gmail.com%3E/2
18.
http://mail-archives.apache.org/mod_mbox/storm-user/201404.mbox/raw/%3CCAJq5PB5pDjYemyf3m+rVyjPzN3_t9_-d8nF-7rWEJVhNXB=y...@mail.gmail.com%3E/3
19.
http://mail-archives.apache.org/mod_mbox/storm-user/201404.mbox/raw/%3CCAJq5PB5pDjYemyf3m+rVyjPzN3_t9_-d8nF-7rWEJVhNXB=y...@mail.gmail.com%3E/4
20. http://www.apache.org/
21. mailto:[email protected]
22. tel:%2B1%20812%20369%206762
23. http://supunk.blogspot.com/
24. http://www.infoobjects.com/
25. tel:%2B1%20%28408%29%20988-2000
26. tel:%2B1%20%28408%29%20716-2726
27. http://www.apache.org/
28. mailto:[email protected]
29. http://supunk.blogspot.com/
