Does the storm share the same zookeeper with Kafka? And where did you find the error?
Regard, Junfeng Chen On Wed, Oct 19, 2016 at 5:30 PM, Daniccan VP <[email protected]> wrote: > > String zookeeperConnectionString = "zookeeper-server01:2181, > zookeeper-server02:2181,zookeeper-server03:2181,zookeeper-server04:2181"; > > BrokerHosts hosts = new ZkHosts(zookeeperConnectionString); > > SpoutConfig spoutConfig = new SpoutConfig(hosts, kafkaTopicName, "/" + > kafkaTopicName, kafkaSpoutConsumerGroupId); > > spoutConfig.bufferSizeBytes = 1024 * 1024 * 5; > spoutConfig.fetchSizeBytes = 1024 * 1024 * 5; > spoutConfig.startOffsetTime = kafka.api.OffsetRequest.EarliestTime(); > spoutConfig.useStartOffsetTimeIfOffsetOutOfRange = true; > > spoutConfig.scheme = new KeyValueSchemeAsMultiScheme(new > CustomizedKeyValueScheme()); > > KafkaSpout kafkaSpout = new KafkaSpout(spoutConfig); > > > Thanks and Regards, > *Daniccan VP **|* *Junior Software Engineer* > Email : [email protected] > ------------------------------ > *From:* [email protected] [[email protected]] on behalf of Junfeng Chen > [[email protected]] > *Sent:* Wednesday, October 19, 2016 2:43 PM > *To:* user@storm apache. org > *Subject:* Re: KafkaSpout offset stored under wrong partition in zookeeper > > Can you provide your KafkaSpout configuration? > > > Regard, > Junfeng Chen > > On Wed, Oct 19, 2016 at 5:07 PM, Daniccan VP <[email protected]> > wrote: > >> Hi, >> >> I am running a Storm Topology with a KafkaSpout which consumes messages >> from Kafka Brokers. The issue I am facing is that the assigned partition >> for a worker process and the partition and offset information of the >> consumer stored in zookeeper are different. >> >> For example, One of the worker processes is assigned with a partition id >> of 1. But, it keeps searching for the last offset under partition 2 in >> Zookeeper path. Has anyone else faced a similar issue ? >> >> Thanks and Regards, >> *Daniccan VP **|* *Junior Software Engineer* >> Email : [email protected] >> ************************************************************ >> ************************************************************ >> ******************************************* This email and any files >> transmitted with it are confidential and intended solely for the use of the >> individual or entity to whom they have been addressed. If you are not the >> intended recipient, you are notified that disclosing, copying, distributing >> or taking any action in reliance on the contents of this information is >> strictly prohibited. Please notify the sender immediately by e-mail if you >> have received this e-mail by mistake and delete this e-mail from your >> system. >> > > ************************************************************ > ************************************************************ > ******************************************* This email and any files > transmitted with it are confidential and intended solely for the use of the > individual or entity to whom they have been addressed. If you are not the > intended recipient, you are notified that disclosing, copying, distributing > or taking any action in reliance on the contents of this information is > strictly prohibited. Please notify the sender immediately by e-mail if you > have received this e-mail by mistake and delete this e-mail from your > system. >
