I’ve implemented a topology consisting of a spout, processing bolt and a sink 
bolt pushing data back to Kafka. 

By examining the logs, I’ve seen that for all of the 12 partitions the 
totalSpoutLag and totalLatestTimeOffset remain constant (i.e. approximately at 
830K), although tuples are sent, and received by a Kafka broker for preserving 
within a log. 

Furthermore, by examining the logs, I’ve seen the following in regard to 
missing tuple IDs:

2016-09-20 18:52:47.382 o.a.s.d.executor [INFO] Execute done TUPLE source: 
kafka_spout:39, stream: default, id: {6264937611590370915=6351865697993896503}, 
[[B@ac8efd] TASK: 23 DELTA: 
2016-09-20 18:52:47.382 o.a.s.d.executor [INFO] BOLT ack TASK: 21 TIME:  TUPLE: 
source: kafka_spout:41, stream: default, id: 
{-5557071446491187803=8360464973979946579}, [[B@647ad40d]
2016-09-20 18:52:47.382 o.a.s.d.executor [INFO] TRANSFERING tuple [dest: 4 
tuple: source: kafka_bolt:35, stream: __metrics, id: {}, 
[#object[org.apache.storm.metric.api.IMetricsConsumer$TaskInfo 0x1bb1b785 
[#object[org.apache.storm.metric.api.IMetricsConsumer$DataPoint 0x2be31af 
"[kafka_bolt_count_metric = 124]"]]]] 

Any clue what might be going wrong? Why might the offset not get updated, and 
further pending tuples processed? 

Thanks in advance! 


Reply via email to