[
https://issues.apache.org/jira/browse/STORM-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16258460#comment-16258460
]
Jungtaek Lim commented on STORM-2824:
-------------------------------------
To understanding exactly once processing, you may want to imagine failure
scenario like worker is suddenly killed.
Suppose tuple is completely processed but worker suddenly killed before
notifying its progress to upstream. How we handle it? If we disregard about
unnoticed thing and don't replay, it looks like "exactly once" in this
scenario. But tweaking the scenario a bit, suppose tuple is in processing and
worker suddenly killed, then we should replay, to guarantee "at least once".
So guaranteeing exactly once is not easy one, and as Stig mentioned, without
relying idempotent writes or other tricks I don't see anything guaranteeing
exactly once for streaming system.
> Ability to configure topologies for exactly once processing
> -----------------------------------------------------------
>
> Key: STORM-2824
> URL: https://issues.apache.org/jira/browse/STORM-2824
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Affects Versions: 1.0.1
> Environment: CentOS 7, Docker
> Reporter: Anton Alfred
> Priority: Minor
>
> The default implementation of a spout (Kafka) is to wait for
> acknowledgement, if an acknowledgement is not provided the tuple is replayed
> leading to an at least once processing model.
> Can an option be provided to always acknowledge even in the event of error in
> any spout or bolt and the user decide which mode the topology should be
> configured.
> There are cases like multiple bolts (B) inserting to persistent stores (PS)
> like B1 - PS1, B2-PS2, B3-PS3, the fact that B2-PS2 bolt fail doesn't mean
> that the tuple needs to be replayed leading to complexity on the logic of
> bolts, it would be easier if this was configurable and the user of the
> topology decides which style to choose.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
