Thanks, Bill, I turn on the debug mode,
by
Config conf = new Config();
conf.setDebug(true);
This is what I get from log:
root@DO-mq-dev:/etc/storm/logs# ll
total 708
drwxr-xr-x 2 root root 4096 Aug 11 13:40 ./
drwxr-xr-x 10 root root 4096 Aug 11 13:33 ../
-rw-r--r-- 1 root root 0 Aug 11 13:33 access.log
-rw-r--r-- 1 root root 0 Aug 11 13:33 metrics.log
-rw-r--r-- 1 root root 407392 Oct 28 16:08 nimbus.log
-rw-r--r-- 1 root root 270708 Oct 28 15:45 supervisor.log
-rw-r--r-- 1 root root 19965 Oct 28 16:02 ui.log
root@DO-mq-dev:/etc/storm/logs# tail nimbus.log
at
org.apache.thrift7.ProcessFunction.process(ProcessFunction.java:32)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
org.apache.thrift7.TBaseProcessor.process(TBaseProcessor.java:34)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
org.apache.thrift7.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:632)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
org.apache.thrift7.server.THsHaServer$Invocation.run(THsHaServer.java:201)
[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_55]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_55]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55]
2014-10-28 16:08:38 o.a.z.ZooKeeper [INFO] Session: 0x14915a47cdde1f1 closed
2014-10-28 16:08:38 o.a.z.ClientCnxn [INFO] EventThread shut down
2014-10-28 16:08:38 b.s.d.nimbus [INFO] Shut down master
This is not correct, since I submit many times topology to cluster, seems
didn't get recorded in log files. Any clues?
thanks
Alec
On Thu, Oct 30, 2014 at 4:20 AM, Brunner, Bill <[email protected]>
wrote:
> Turn on debug mode and tail the log
>
>
>
> *From:* Sa Li [mailto:[email protected]]
> *Sent:* Wednesday, October 29, 2014 8:13 PM
> *To:* [email protected]
> *Subject:* Re: not writing data into DB in storm cluster, but does in
> localcluster
>
>
>
> I compile the code as
>
> mvn clean package -P cluster
>
>
>
> and run as
>
>
>
> storm jar
> target/kafka-storm-ingress-0.0.1-SNAPSHOT-jar-with-dependencies.jar
> storm.ingress.KafkaIngressTopology
>
>
>
> it running, but nothing populated into DB, how can debug it in cluster
> mode?
>
>
>
> thanks
>
>
>
> Alec
>
>
>
> On Wed, Oct 29, 2014 at 2:24 PM, Sa Li <[email protected]> wrote:
>
> Thanks for reply Bill, here how I submit it :
>
>
>
> if (args != null && args.length > 0) {
>
> System.out.println("local mode....");
>
> cluster.submitTopology("topictest", conf,
> buildTridentKafkaTopology());
>
> Thread.sleep(1500);
>
>
>
> //cluster.shutdown();
>
> //drpc.shutdown();
>
> }
>
> else {
>
> System.out.println("Storm cluster....");
>
> conf.setNumWorkers(10);
>
> StormSubmitter.submitTopology("topictest", conf,
> buildTridentKafkaTopology());
>
> }
>
>
>
>
>
> So I get the number of Workers as 10.
>
>
>
>
>
> Thanks
>
>
>
> Alec
>
>
>
> On Wed, Oct 29, 2014 at 2:16 PM, Brunner, Bill <[email protected]>
> wrote:
>
> Do you have least 1 Worker defined in your topology?
>
>
>
> *From:* Sa Li [mailto:[email protected]]
> *Sent:* Wednesday, October 29, 2014 5:08 PM
> *To:* [email protected]
> *Subject:* not writing data into DB in storm cluster, but does in
> localcluster
>
>
>
> Hi, All
>
>
>
> I am running a kafkaSpout to consume data from kafka and write data into
> postgresql DB, it works in localcluster even it is slow (we need to
> diagnose what the problem is). When I sumbitted it in storm cluster, it
> doesn't show exceptions, and I see the topology is alive in StormUI, but
> just no data being written into DB, why that happen?
>
>
>
> thanks
>
>
>
> Alec
> ------------------------------
>
> This message, and any attachments, is for the intended recipient(s) only,
> may contain information that is privileged, confidential and/or proprietary
> and subject to important terms and conditions available at
> http://www.bankofamerica.com/emaildisclaimer. If you are not the intended
> recipient, please delete this message.
>
>
>
>
> ------------------------------
> This message, and any attachments, is for the intended recipient(s) only,
> may contain information that is privileged, confidential and/or proprietary
> and subject to important terms and conditions available at
> http://www.bankofamerica.com/emaildisclaimer. If you are not the intended
> recipient, please delete this message.
>
