[
https://issues.apache.org/jira/browse/KAFKA-8650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884826#comment-16884826
]
Richard Yu commented on KAFKA-8650:
-----------------------------------
[~mjsax] Do you think this ticket is something that Kafka commuity would need
within the next release? Would be good to get a gauge on this issue's priority.
> Streams does not work as expected with auto.offset.reset=none
> -------------------------------------------------------------
>
> Key: KAFKA-8650
> URL: https://issues.apache.org/jira/browse/KAFKA-8650
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Affects Versions: 2.3.0
> Reporter: Raman Gupta
> Priority: Major
> Labels: needs-kip
>
> The auto.offset.reset policy of none is useful as a safety measure,
> especially when
> * exactly-once processing is desired, or
> * at-least-once is desired, but it is expensive to reprocess from the
> beginning.
> In this case, using "none" forces the ops team to explicitly set the offset
> before the stream can re-start processing, in the (hopefully rare) situations
> in which the stream consumer offset has been lost for some reason, or in the
> case of a new stream that should not start processing from the beginning or
> the end, but somewhere in the middle (this scenario might occur during topic
> migrations).
> Kafka streams really only supports auto.offset.reset of earliest or latest
> (see the `Topology.AutoOffsetReset` enum). It is also possible to use the
> auto.offset.reset configuration value, but this works suboptimally because if
> the streams application reset tool is used (even with a specific offset
> specified), the offset is set for the input topic, but it is not, and cannot
> be, set for the internal topics, which won't exist yet.
> The internal topics are created by Kafka streams at startup time, but because
> the auto.offset.reset policy of "none" is passed to the consumer of those
> internal topics, the Kafka stream fails to start with a
> "NoOffsetForPartitionException".
> Proposals / options:
> 1) Allow auto.offset.reset=none to be specified in Consumed.with() so that it
> affects the input topics, but not the internal topics.
> 2) Allow streams to be configured with auto.offset.reset=none, but explicitly
> set the offset to 0 for newly created internal topics.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
