[
https://issues.apache.org/jira/browse/KAFKA-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17329888#comment-17329888
]
A. Sophie Blee-Goldman commented on KAFKA-10493:
------------------------------------------------
Imo we should find a way to fix this that doesn't prevent users from leveraging
the source topic optimization. As I've mentioned before, the additional storage
footprint from changelogs is a very real complaint and has been cited as the
reason for not using Kafka Streams in the past. And it sounds to me like this
would make it even worse, as we would need to not only use a dedicated
changelog for all source KTables but also disable compaction entirely IIUC.
That just does not sound like a feasible path forward
I haven't fully digested this current discussion about the impact of dropping
out-of-order updates with a compacted changelog, but perhaps we could store
some information in the committed offset metadata to help us here?
> KTable out-of-order updates are not being ignored
> -------------------------------------------------
>
> Key: KAFKA-10493
> URL: https://issues.apache.org/jira/browse/KAFKA-10493
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.6.0
> Reporter: Pedro Gontijo
> Assignee: Matthias J. Sax
> Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: KTableOutOfOrderBug.java
>
>
> On a materialized KTable, out-of-order records for a given key (records which
> timestamp are older than the current value in store) are not being ignored
> but used to update the local store value and also being forwarded.
> I believe the bug is here:
> [https://github.com/apache/kafka/blob/2.6.0/streams/src/main/java/org/apache/kafka/streams/state/internals/ValueAndTimestampSerializer.java#L77]
> It should return true, not false (see javadoc)
> The bug impacts here:
> [https://github.com/apache/kafka/blob/2.6.0/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableSource.java#L142-L148]
> I have attached a simple stream app that shows the issue happening.
> Thank you!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
