Also I am testing the KafkaSpout throughput, running KafkaSpout on local mode, it consumes the data, (i retrieve the data and write into DB), but when I am checking the kafkaMoniter UI, it is the kafkaOffsetMonitering, I click the topic "topictest" which I generate the stream in kafkaproducer, but it unable to find active consumers, I feel like the kafka and storm have problem to integrate (maybe I am wrong). Or the conflict on zk server, since I believe the UIs get stats from ZK server.
Thanks Alec On Thu, Oct 30, 2014 at 10:14 AM, Sa Li <[email protected]> wrote: > Hi, All > > I have been working Storm for a while, mostly I work on localmode, but > recently I want to test the thoughtput of KafkaSpout, when I submit the > topology to cluster, it didn't work while working fine in local mode. > > Test the cluster, I run the example typologies come with Storm package in > cluster, it is running but I can't see any throughput in storm UI (see the > attached screenshot), is that correct? if not, is it the problem of my > storm setup? > > I am setup the storm in a single server, using supervisord to run storm, > here is the storm.yaml > > ########### These MUST be filled in for a storm configuration > storm.zookeeper.servers: > - "10.100.70.128" > # - "server2" > # > > storm.zookeeper.port: 2181 > > nimbus.host: "10.100.70.128" > > storm.local.dir: "/app/storm" > > java.library.path: "/usr/lib/jvm/java-7-openjdk-amd64" > > supervisor.slots.ports: > > - 6700 > - 6701 > - 6702 > - 6703 > > drpc.servers: > - "10.100.70.128" > # - "server2" > > nimbus.childopts: "-Xms2048m" > ui.childopts: "-Xms1024m" > logviewer.childopts: "-Xmx512m" > supervisor.childopts: "-Xms1024m" > topology.trident.batch.emit.interval.millis: 100 > topology.message.timeout.secs: 300 > > > thanks > > Alec > >
