Re: is there a way for kafka spout to read from written zookeeper offset

[이승진]

thanks for comment.
 
even though I restart topology and obviously there exists offset information in 
zookeeper,  "No partition information found, using configuration to determine 
offset" happens. 
 
I'm using spoutConfig.forceFromStart = true; is this option has something to do 
with that? what exactly does that option mean?
 
 
 
-----Original Message-----
From: "Harsha"<st...@harsha.io> 
To: <user@storm.apache.org>; 
Cc: 
Sent: 2014-12-02 (화) 02:27:58
Subject: Re: is there a way for kafka spout to read from written zookeeper 
offset







KafkaSpout will read  last committed offset from zookeeper. If there is no 
offset in the zookeeper than only it will use configured startOffsetTime.  Can 
you check in your worker logs after restarting your topology for this message 
"No partition information found, using configuration to determine offset"  
-Harsha
 
 
On Mon, Dec 1, 2014, at 01:48 AM, 이승진 wrote: 
Hi all, 
 
Kafkaspout periodically write each partition offset to zookeeper. 
 
and spoutConfig.startOffsetTime=-2 means from the beginning, -1 from the latest 
offset. 
 
Is there a option to read from last committed consumer offset? 
 
When I restart topology due to bugfix or whatsoever, some of the messages are 
not processed by storm because I set it to -1. 
 
Sincerely