Yes, I was able to see the topology in Storm UI and nothing was logged into worker logs. However, as I mentioned, I am able to resolve it by finding an hint in supervisor.log file this time.
On Mon, Aug 25, 2014 at 8:58 PM, Georgy Abraham <[email protected]> wrote: > Are you able to see the topology in storm UI or with storm list command ?? > And worker mentioned in the UI doesn't have any log ?? > ------------------------------ > From: Vikas Agarwal > Sent: 25-08-2014 PM 05:25 > To: [email protected] > Subject: Storm not processing topology without logs > > > Hi, > > I have started to explore the Storm for distributed processing for our use > case which we were earlier fulfilling by JMS based MQ system. Topology > worked after some efforts. It has one spout (KafkaSpout from kafka-storm > project) and 3 bolts. First bolt sets context for other two bolts which in > turn do some processing on the tuples and persist the analyzed results in > some DB (Mongo, Solr, HBase etc). > > Recently the topology stopped working. I am able to submit the topology > and it does not throw any error in submitting the topology, however, > nimbus.log or worker-6701.log files are not showing any progress and > eventually topology does not consume any message. I don't have doubt on > KafkaSpout because if it was the culprit, at least some initialization logs > of spout and bolts should have been there in nimbus.log or worker-xxxx.log. > Isn't it? > > Here is the snippet of nimbus.log after uploading the jar to cluster > > Uploading file from client to > /hadoop/storm/nimbus/inbox/stormjar-31fe068b-337b-428f-8ae2-fe13c706b2ab.jar > 2014-08-25 07:07:49 b.s.d.nimbus [INFO] Finished uploading file from > client: > /hadoop/storm/nimbus/inbox/stormjar-31fe068b-337b-428f-8ae2-fe13c706b2ab.jar > 2014-08-25 07:07:49 b.s.d.nimbus [INFO] Received topology submission for > aleads with conf {"topology.max.task.parallelism" nil, > "topology.acker.executors" nil, "topology.kryo.register" nil, > "topology.kryo.decorators" (), "topology.name" "aleads", "storm.id" > "aleads-3-1408964869", "modelId" "ut", "topology.workers" 1, > "topology.debug" true} > 2014-08-25 07:07:50 b.s.d.nimbus [INFO] Activating aleads: > aleads-3-1408964869 > 2014-08-25 07:07:50 b.s.s.EvenScheduler [INFO] Available slots: > (["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701] > ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6700]) > 2014-08-25 07:07:50 b.s.d.nimbus [INFO] Setting new assignment for > topology id aleads-3-1408964869: > #backtype.storm.daemon.common.Assignment{:master-code-dir > "/hadoop/storm/nimbus/stormdist/aleads-3-1408964869", :node->host > {"e56c2cc7-d35a-4355-9906-506618ff70c5" "hdp.ambari"}, :executor->node+port > {[2 2] ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [3 3] > ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [4 4] > ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [5 5] > ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [6 6] > ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [7 7] > ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [8 8] > ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [9 9] > ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [1 1] > ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701]}, :executor->start-time-secs > {[1 1] 1408964870, [9 9] 1408964870, [8 8] 1408964870, [7 7] 1408964870, [6 > 6] 1408964870, [5 5] 1408964870, [4 4] 1408964870, [3 3] 1408964870, [2 2] > 1408964870}} > > Can anyone guess what I have done wrong and why Storm is not giving any > error log anywhere. > > Storm version is 0.9.1.2.1.3.0-563 (Installed via HortonWorks) > Kafka version is 2.10-0.8.1.1 > Storm-Kafka version 0.9.2-incubating > > -- > Regards, > Vikas Agarwal > 91 – 9928301411 > > InfoObjects, Inc. > Execution Matters > http://www.infoobjects.com > 2041 Mission College Boulevard, #280 > Santa Clara, CA 95054 > +1 (408) 988-2000 Work > +1 (408) 716-2726 Fax > > -- Regards, Vikas Agarwal 91 – 9928301411 InfoObjects, Inc. Execution Matters http://www.infoobjects.com 2041 Mission College Boulevard, #280 Santa Clara, CA 95054 +1 (408) 988-2000 Work +1 (408) 716-2726 Fax
