There was a mistake in my code and now the topology is using the external zookeeper. But I still see the message No partition information found, using configuration to determine offset
I don't know the significance of this message. My main objective is to find out why kafka spout is so slow. thanks Clay On Wed, Feb 4, 2015 at 4:04 AM, clay teahouse <[email protected]> wrote: > Hi, > Even if I use the local cluster with the constructor that specified the > external zookeeper's address, I still see: > backtype.storm.zookeeper - Starting inprocess zookeeper at port 2000 > and > org.apache.storm.zookeeper.ZooKeeper - Initiating client connection, > connectString=localhost:2000 sessionTimeout=20000 > watcher=org.apache.storm.curator.ConnectionState@7887fb2f > > And I still see the message: > > No partition information found, using configuration to determine offset > > I am mostly concerned about the latter message. Why am I getting this > message? > > thanks, > Clay > > > On Tue, Feb 3, 2015 at 11:12 PM, Harsha <[email protected]> wrote: > >> LocalCluster should be used for debugging a topology . There is another >> constructor you can use >> LocalCluster cluster = new LocalCluster("localhost", new Long(2182)); >> first param is zookeeper host and second is the port. >> -Harsha >> >> On Tue, Feb 3, 2015, at 07:48 PM, Shivendra Singh wrote: >> > Hi Clay, >> > >> > The zookeeper info provided to KafkaSpout is external zookeeper which is >> > used by Kafka and where all the host/partition information for Kafka is >> > stored. >> > >> > When you run storm in local mode, it spins up an in-memory zookeeper for >> > storm where it keeps data related to nimbus/supervisors, etc. >> > >> > Thanks, >> > Shivendra >> > >> > >> > > On Feb 3, 2015, at 7:23 PM, clay teahouse <[email protected]> >> wrote: >> > > >> > > Hi, >> > > >> > > I have a topology running in local mode. The topology uses kafkaspout >> and is configured to use the external zookeeper. But when I start the >> topology, I see the following: >> > > >> > > org.apache.storm.zookeeper.ZooKeeper - Initiating client connection, >> connectString=localhost:2000 >> > > >> > > I also messages like the following: >> > > >> > > No partition information found, using configuration to determine >> offset >> > > >> > > I am not sure if these two messages are related. >> > > >> > > Questions: >> > > 1) Why storm is employing the in-memory zookeeper when I have >> configured the spout to use the external zookeeper. >> > > 2)How do I force storm to use the external zookeeper? >> > > 3) Why am I getting the message No partition information found, using >> configuration to determine offset? What configuration is it referring to? >> > > >> > > thanks, >> > > Clay >> > > >> > >
