nicktelford opened a new pull request, #12356: URL: https://github.com/apache/kafka/pull/12356
Instead of emitting a warning when adding out-of-order (by timestamp) records to a `KTable`, we should discard these records silently. Unfortunately, this creates a problem for `KTable`s produced from a source-topic that has been optimized with the source-topic optimization, i.e. when using `StreamsBuilder#table(...)`. If a user topic that is being used as a changelog (by the source-topic optimiation) has compaction enabled, it's possible that the topic-compaction will eliminate newer (by timestamp) records, because compaction is (currently) offset-based. To ensure we don't produce a `KTable` that is inconsistent with the source-topic in this scenario, we must continue to add all records from these topics in to the materialized store, irrespective of timestamp. To facilitate this, `KTableSource` is now explicitly instructed by the topology when it's processing records from an optimized source-topic, via the new `sourceTopicOptimized` field. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
