Stephan Ewen created FLINK-2393:
-----------------------------------
Summary: Add a "stateless at-least-once" mode for streaming
Key: FLINK-2393
URL: https://issues.apache.org/jira/browse/FLINK-2393
Project: Flink
Issue Type: New Feature
Components: Streaming
Affects Versions: 0.10
Reporter: Stephan Ewen
Currently, the checkpointing mechanism provides "exactly once" guarantees. Part
of that is the step that temporarily "aligns" the data streams. This step
increases the tuple latency temporarily.
By offering a version that does not provide "exactly-once", but only
"at-least-once", we can avoid the latency increase. For super-low-latency
applications, that tolerate duplicates, this may be an interesting option.
To realize that, we would use a slightly modified version of the checkpointing
algorithm. Effectively, the streams would not be aligned, but tasks would only
count the received barriers and emit their own barrier as soon as the saw a
barrier from all inputs.
My feeling is that it makes not sense to implement state backups, when being
concerned with this super low latency. The mode would hence be a purely
stateless at-least-once mode.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
