H,
I’ve implemented a topology consisting of a spout, processing bolt and a sink
bolt pushing data back to Kafka.
By examining the logs, I’ve seen that for all of the 12 partitions the
totalSpoutLag and totalLatestTimeOffset remain constant (i.e. approximately at
830K), although tuples are sent, and received by a Kafka broker for preserving
within a log.
Furthermore, by examining the logs, I’ve seen the following in regard to
missing tuple IDs:
2016-09-20 18:52:47.382 o.a.s.d.executor [INFO] Execute done TUPLE source:
kafka_spout:39, stream: default, id: {6264937611590370915=6351865697993896503},
[[B@ac8efd] TASK: 23 DELTA:
2016-09-20 18:52:47.382 o.a.s.d.executor [INFO] BOLT ack TASK: 21 TIME: TUPLE:
source: kafka_spout:41, stream: default, id:
{-5557071446491187803=8360464973979946579}, [[B@647ad40d]
2016-09-20 18:52:47.382 o.a.s.d.executor [INFO] TRANSFERING tuple [dest: 4
tuple: source: kafka_bolt:35, stream: __metrics, id: {},
[#object[org.apache.storm.metric.api.IMetricsConsumer$TaskInfo 0x1bb1b785
"org.apache.storm.metric.api.IMetricsConsumer$TaskInfo@1bb1b785"]
[#object[org.apache.storm.metric.api.IMetricsConsumer$DataPoint 0x2be31af
"[kafka_bolt_count_metric = 124]"]]]]
Any clue what might be going wrong? Why might the offset not get updated, and
further pending tuples processed?
Thanks in advance!
Dominik
