Thanks, Kishore, I am running on local mode, i understand there something must be wrong with offset, here I want consume one topic in kafka which has 1000000 records, i am not sure what else i can do for offset thing?
thanks > On Aug 25, 2015, at 9:16 PM, Kishore Senji <[email protected]> wrote: > > Regarding the partition manager, what it means is that your messages are > failing and Storm is not retrying those because the current offset is atleast > 100000 more than the failed offset. This means some messages are failing to > be processed. Are you having a bigger spout pending? Are you seeing a bigger > failure rate - Check these metrics in Storm UI > > On Tue, Aug 25, 2015 at 4:38 PM, Alec Lee <[email protected] > <mailto:[email protected]>> wrote: > > Hi, All > > I am having such error when I am consuming the data from kafka, the > timestamps are printed by my bolts, which is OK, but error-path, and > partitionManager error seem to be annoying. > Any idea? > > 2013-03-21 12:58:00-07:00 > > > 13486 [ProcessThread(sid:0 cport:-1):] INFO > org.apache.storm.zookeeper.server.PrepRequestProcessor - Got user-level > KeeperException when processing sessionid > :0x14f672ef567000e type:create cxid:0x3 zxid:0x2b txntype:-1 reqpath:n/a > Error Path:/test-topic/653005be-22a6-46ab-8f46-beee1bcff5f4 > Error:KeeperErrorCode = N > oNode for /test-topic/653005be-22a6-46ab-8f46-beee1bcff5f4 > > > 2013-03-21 12:21:00-07:00 > 2013-03-21 13:00:00-07:00 > 2013-03-21 11:47:00-07:00 > 2013-03-21 13:01:00-07:00 > 2013-03-21 11:52:00-07:00 > 2013-03-21 12:02:00-07:00 > 2013-03-21 13:02:00-07:00 > 2013-03-21 12:22:00-07:00 > 2013-03-21 12:23:00-07:00 > 2013-03-21 13:03:00-07:00 > 2013-03-21 13:05:00-07:00 > 2013-03-21 13:04:00-07:00 > 69074 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping failed > tuple at offset=10083 because it's more than maxOffsetBehind=100000 behind > _emitt > edToOffset=127693 > 69074 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping failed > tuple at offset=25687 because it's more than maxOffsetBehind=100000 behind > _emitt > edToOffset=127693 > 69074 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping failed > tuple at offset=23152 because it's more than maxOffsetBehind=100000 behind > _emitt > edToOffset=127693 > 69074 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping failed > tuple at offset=3343 because it's more than maxOffsetBehind=100000 behind > _emitte > dToOffset=127693 > 69074 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping failed > tuple at offset=3367 because it's more than maxOffsetBehind=100000 behind > _emitte > dToOffset=127693 > 69075 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping failed > tuple at offset=8833 because it's more than maxOffsetBehind=100000 behind > _emitte > dToOffset=127693 > 69078 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping failed > tuple at offset=7219 because it's more than maxOffsetBehind=100000 behind > _emitte > dToOffset=127693 > 69078 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping failed > tuple at offset=22552 because it's more than maxOffsetBehind=100000 behind > _emitt > edToOffset=127693 > 69078 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping failed > tuple at offset=23607 because it's more than maxOffsetBehind=100000 behind > _emitt > edToOffset=127693 > 69078 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping failed > tuple at offset=8356 because it's more than maxOffsetBehind=100000 behind > _emitte > dToOffset=127693 > > > Thanks > Al >
