[
https://issues.apache.org/jira/browse/KAFKA-9224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17086127#comment-17086127
]
Boyang Chen commented on KAFKA-9224:
------------------------------------
This basically goes back to what consistency guarantee for IQ queries we are
providing. Consider a billing pipeline, which doesn't turn on EOS and only rely
on IQ, then our commitment as at_least_once doesn't guarantee the ordering for
all times, so it's possible to see non-monotonic increasing cost value over
time. These guarantees are protected by EOS, which aims for not just one time
processing but also strong ordering guarantee.
Of course we could disagree here, but as long as unclean leader election is
still possible, I don't think a fail-over task could provide any linearizable
guarantee towards to previous owner. It's good for us to clarify on the SLA
first if we don't think this is expected for non-EOS.
> State store should not see uncommitted transaction result
> ---------------------------------------------------------
>
> Key: KAFKA-9224
> URL: https://issues.apache.org/jira/browse/KAFKA-9224
> Project: Kafka
> Issue Type: Improvement
> Reporter: Boyang Chen
> Assignee: Boyang Chen
> Priority: Major
>
> Currently under EOS, the uncommitted write could be reflected in the state
> store before the ongoing transaction is finished. This means interactive
> query could see uncommitted data within state store which is not ideal for
> users relying on state stores for strong consistency. Ideally, we should have
> an option to include state store commit as part of ongoing transaction,
> however an immediate step towards a better reasoned system is to `write after
> transaction commit`, which means we always buffer data within stream cache
> for EOS until the ongoing transaction is committed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
