I am using apache-storm-0.9.2-incubating and use the built in kafka spout. I do not see any errors in worker logs. I got bunch of worker logs and checked couple of them and dont see problems.
-- Kushan Maskey 817.403.7500 M. Miller & Associates <http://mmillerassociates.com/> [email protected] On Sun, Nov 9, 2014 at 9:34 AM, Harsha <[email protected]> wrote: > Kushan, > Which KafkaSpout are you using. Do you see any errors in worker > logs . > -Harsha > > > On Sun, Nov 9, 2014, at 07:22 AM, Kushan Maskey wrote: > > THanks Harsha, here they are, > > > # Licensed to the Apache Software Foundation (ASF) under one > > # or more contributor license agreements. See the NOTICE file > > # distributed with this work for additional information > > # regarding copyright ownership. The ASF licenses this file > > # to you under the Apache License, Version 2.0 (the > > # "License"); you may not use this file except in compliance > > # with the License. You may obtain a copy of the License at > > # > > # http://www.apache.org/licenses/LICENSE-2.0 > > # > > # Unless required by applicable law or agreed to in writing, software > > # distributed under the License is distributed on an "AS IS" BASIS, > > # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. > > # See the License for the specific language governing permissions and > > # limitations under the License. > > > ########### These MUST be filled in for a storm configuration > > # storm.zookeeper.servers: > > # - "server1" > > # - "server2" > > storm.zookeeper.servers: > > - "nmcxstrmd001" > > > storm.local.dir: "/data/disk00/storm/localdir" > > > nimbus.host: "nmcxstrmd001" > > > supervisor.slots.ports: > > - 6700 > > - 6701 > > - 6702 > > - 6703 > > - 6704 > > - 6705 > > - 6706 > > - 6707 > > - 6708 > > - 6709 > > - 6710 > > - 6711 > > - 6712 > > - 6713 > > - 6714 > > - 6715 > > - 6716 > > - 6717 > > - 6718 > > - 6719 > > - 6720 > > - 6721 > > - 6722 > > - 6723 > > - 6724 > > - 6725 > > - 6726 > > - 6727 > > - 6728 > > supervisor.childopts: "-Xmx256m" > > worker.childopts: "-Xmx1024m" > > #how long supervisor will wait to ensure that a worker process is started > > supervisor.worker.start.timeout.secs: 120 > > #how long between heartbeats until supervisor considers that worker dead > and tries to restart it > > supervisor.worker.timeout.secs: 30 > > #how frequently the supervisor checks on the status of the processes it's > monitoring and restarts if necessary > > supervisor.monitor.frequency.secs: 3 > > #how frequently the supervisor heartbeats to the cluster state (for nimbus) > > supervisor.heartbeat.frequency.secs: 5 > > supervisor.enable: true > > > # > > # nimbus.host: "nimbus" > > # > > # > > # ##### These may optionally be filled in: > > # > > ## List of custom serializations > > # topology.kryo.register: > > # - org.mycompany.MyType > > # - org.mycompany.MyType2: org.mycompany.MyType2Serializer > > # > > ## List of custom kryo decorators > > # topology.kryo.decorators: > > # - org.mycompany.MyDecorator > > # > > ## Locations of the drpc servers > > # drpc.servers: > > # - "server1" > > # - "server2" > > > ## Metrics Consumers > > # topology.metrics.consumer.register: > > # - class: "backtype.storm.metric.LoggingMetricsConsumer" > > # parallelism.hint: 1 > > # - class: "org.mycompany.MyMetricsConsumer" > > # parallelism.hint: 1 > > # argument: > > # - endpoint: "metrics-collector.mycompany.org" > > > > and storm config properties taht I read from external file: > > > zookeeper.server.host=zkserver123 > > zookeeper.server.port=2181 > > zookeeper.group.id=consumer_group > > zookeeper.session.timeout.ms=400 > > zookeeper.auto.commit.enable=false > > zookeeper.consumer.timeout.ms=500 > > > zookeeper.auto.offset.reset=smallest > > kafka.topic.rtSalesMQ=my_topic_on_kafka > > > > storm.config.setMaxTaskParallelism=1 > > storm.config.setNumWorkers=4 > > storm.config.setMaxSpoutPending=4 > > storm.config.numAckers=1 > > storm.config.parallelism.hint.bolt=3 > > storm.config.parallelism.hint.spout=4 > > > storm.config.receiver.buffer.size=32 > > storm.config.transfer.buffer.size=32 > > # 1024*64 = 65536 > > storm.config.executer.receiver.buffer.size=65536 > > > storm.config.executer.transfer.buffer.size=65536 > > > LMK if there is anything that is not correct. Thanks. > > -- > Kushan Maskey > 817.403.7500 > M. Miller & Associates <http://mmillerassociates.com/> > [email protected] > > On Sun, Nov 9, 2014 at 9:15 AM, Harsha <[email protected]> wrote: > > > Kushan, > Please share your storm.yaml and your KafkaSpout config. > Thanks, > Harsha > > > On Sun, Nov 9, 2014, at 07:13 AM, Kushan Maskey wrote: > > I have a topology that runs and reads message from a kafka spout > successfully. But when i deploy the same topology and run it on a server it > does not read any message. This code use to run fine earlier like 2 days > before. I am using same configure parameters properties as in the local. I > am stuck and need help asap. Thank you in advance. > > -- > Kushan Maskey > 817.403.7500 > M. Miller & Associates <http://mmillerassociates.com/> > [email protected] > > > > > > >
