Santosh,
Make sure your SpoutConfig(BrokerHosts hosts, String topic,
String zkRoot, String id); "String id" is static value and
unique across the topologies. If you killing & re-deploying
the topology the value shouldn't be changing. Apart from that
depending on which version you are using forceFromStart or
ignoreZkOffsets set to false.
-Harsha
On Sun, Oct 25, 2015, at 06:18 AM, Craig Charleton wrote:
> Keep in mind that Zookeeper stores the Kafka offsets as they relate to
> the consumer group, not overall. Maybe you have another component running
> under the same consumer group or you could experiment with running your
> topology with a different Kafka consumer group.
>
> Sent from my iPhone
>
> > On Oct 25, 2015, at 2:16 AM, Santosh Pingale <[email protected]>
> > wrote:
> >
> > --089e01184bf256960a0522e7cb6c
> > Content-Type: text/plain; charset=UTF-8
> >
> > I am facing a very weird issue at the moment.
> >
> > I have a storm topology which uses Kafka Spout and it stores offsets in ZK.
> >
> > Kafka topic sometimes do not receive messages for a brief period of time
> > and the topology catches up with the Kafka offsets.
> >
> > But after sometime, offset stored in ZK change to a lesser number
> > indicating topology is failing behind. I need to restart the topology to
> > make it catch up again. I am not able to understand whats going on. This is
> > very strange. I believe there is no other topology running with same ZK
> > root.
> >
> > Storm Version: 0.9.5
> > ZK Version: 3.4.6
> > Kafka Version : 0.8.1.1.
> > Storm-Kafka Version: 0.9.5
> >
> > Your help is appreciated.
> >
> > --089e01184bf256960a0522e7cb6c
> > Content-Type: text/html; charset=UTF-8
> > Content-Transfer-Encoding: quoted-printable
> >
> > <div dir=3D"ltr"><div>I am facing a very weird issue at the moment.</div><d=
> > iv><br></div><div>I have a storm topology which uses Kafka Spout and it sto=
> > res offsets in ZK.</div><div><br></div><div>Kafka topic sometimes do not re=
> > ceive messages for a brief period of time and the topology catches up with =
> > the Kafka offsets.=C2=A0</div><div><br></div><div>But after sometime, offse=
> > t stored in ZK change to a lesser number indicating topology is failing beh=
> > ind. I need to restart the topology to make it catch up again. I am not abl=
> > e to understand whats going on. This is very strange. I believe there is no=
> > other topology running with same ZK root.</div><div><br></div><div>Storm V=
> > ersion: 0.9.5</div><div>ZK Version: 3.4.6</div><div>Kafka Version : 0.8.1.1=
> > .</div><div>Storm-Kafka Version: 0.9.5</div><div><br></div><div>Your help i=
> > s appreciated.</div></div>
> >
> > --089e01184bf256960a0522e7cb6c--
