[
https://issues.apache.org/jira/browse/STORM-2648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stig Rohde Døssing updated STORM-2648:
--------------------------------------
Fix Version/s: (was: 1.1.1)
> Kafka spout can't show acks/fails and complete latency when auto commit is
> enabled
> ----------------------------------------------------------------------------------
>
> Key: STORM-2648
> URL: https://issues.apache.org/jira/browse/STORM-2648
> Project: Apache Storm
> Issue Type: New Feature
> Components: storm-kafka-client
> Affects Versions: 2.0.0, 1.1.0, 1.2.0
> Reporter: Stig Rohde Døssing
> Priority: Minor
>
> The storm-kafka-client spout currently emits tuples with no message ids if
> auto commit is enabled. This causes the ack/fail/complete latency counters in
> Storm UI to be 0. In some cases this is desirable because the user may not
> care, and doesn't want the overhead of Storm tracking tuples.
> [~avermeerbergen] expressed a desire to be able to use auto commit without
> these counters being disabled, presumably to monitor topology performance.
> In order to support having acks/fails and complete latency when using auto
> commit, we need to make the spout always emit tuples with anchors. Since the
> spout doesn't care about acks or fails when auto commit is enabled, we should
> make the spout ack/fail methods return immediately if auto commit is enabled.
> I think we don't lose much by switching to this approach. Users that want the
> statistics in Storm UI can enable auto commit and leave topology ackers at
> some non-zero number. Users that don't care and don't want the overhead of
> having Storm track the tuples can set topology ackers to 0, which should make
> the spout behave a lot like it does now.
> The only use case I can think of where this won't work is if someone with
> multiple spouts in a topology needs one to be auto commit and one to be
> at-least-once, and they can't live with the overhead of tuple tracking for
> the auto commit spout. If this is a real use case, it can be worked around
> with an extra configuration parameter to switch whether tuples are emitted
> unanchored, but I'd rather keep it simple for now.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
