Regarding the partition manager, what it means is that your messages are failing and Storm is not retrying those because the current offset is atleast 100000 more than the failed offset. This means some messages are failing to be processed. Are you having a bigger spout pending? Are you seeing a bigger failure rate - Check these metrics in Storm UI
On Tue, Aug 25, 2015 at 4:38 PM, Alec Lee <[email protected]> wrote: > > Hi, All > > I am having such error when I am consuming the data from kafka, the > timestamps are printed by my bolts, which is OK, but error-path, and > partitionManager error seem to be annoying. > Any idea? > > 2013-03-21 12:58:00-07:00 > > > 13486 [ProcessThread(sid:0 cport:-1):] INFO > org.apache.storm.zookeeper.server.PrepRequestProcessor - Got user-level > KeeperException when processing sessionid > :0x14f672ef567000e type:create cxid:0x3 zxid:0x2b txntype:-1 reqpath:n/a > Error Path:/test-topic/653005be-22a6-46ab-8f46-beee1bcff5f4 > Error:KeeperErrorCode = N > oNode for /test-topic/653005be-22a6-46ab-8f46-beee1bcff5f4 > > > 2013-03-21 12:21:00-07:00 > 2013-03-21 13:00:00-07:00 > 2013-03-21 11:47:00-07:00 > 2013-03-21 13:01:00-07:00 > 2013-03-21 11:52:00-07:00 > 2013-03-21 12:02:00-07:00 > 2013-03-21 13:02:00-07:00 > 2013-03-21 12:22:00-07:00 > 2013-03-21 12:23:00-07:00 > 2013-03-21 13:03:00-07:00 > 2013-03-21 13:05:00-07:00 > 2013-03-21 13:04:00-07:00 > 69074 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping > failed tuple at offset=10083 because it's more than maxOffsetBehind=100000 > behind _emitt > edToOffset=127693 > 69074 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping > failed tuple at offset=25687 because it's more than maxOffsetBehind=100000 > behind _emitt > edToOffset=127693 > 69074 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping > failed tuple at offset=23152 because it's more than maxOffsetBehind=100000 > behind _emitt > edToOffset=127693 > 69074 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping > failed tuple at offset=3343 because it's more than maxOffsetBehind=100000 > behind _emitte > dToOffset=127693 > 69074 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping > failed tuple at offset=3367 because it's more than maxOffsetBehind=100000 > behind _emitte > dToOffset=127693 > 69075 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping > failed tuple at offset=8833 because it's more than maxOffsetBehind=100000 > behind _emitte > dToOffset=127693 > 69078 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping > failed tuple at offset=7219 because it's more than maxOffsetBehind=100000 > behind _emitte > dToOffset=127693 > 69078 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping > failed tuple at offset=22552 because it's more than maxOffsetBehind=100000 > behind _emitt > edToOffset=127693 > 69078 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping > failed tuple at offset=23607 because it's more than maxOffsetBehind=100000 > behind _emitt > edToOffset=127693 > 69078 [Thread-14-words] INFO storm.kafka.PartitionManager - Skipping > failed tuple at offset=8356 because it's more than maxOffsetBehind=100000 > behind _emitte > dToOffset=127693 > > > Thanks > Al >
