[
https://issues.apache.org/jira/browse/KAFKA-19775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikita Shupletsov updated KAFKA-19775:
--------------------------------------
Description:
If there is a KS application that uses a regex source, and we create a new
topic that matches that regex, but produce no messages, the application will
get into an {{ERROR}} state.
if we take
{}[RegexSourceIntegrationTest#testRegexRecordsAreProcessedAfterNewTopicCreatedWithMultipleSubtopologies|#L206{}}}],
but without producing any messages to {{TEST-TOPIC-2}} the problem will
reproduce:
{quote}{{org.apache.kafka.streams.errors.StreamsException:
java.lang.IllegalStateException: Stream task 0_0 does not know the partition:
TEST-TOPIC-2-0}}
{{ at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:981)}}
{{ at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:898)}}
{{Caused by: java.lang.IllegalStateException: Stream task 0_0 does not know the
partition: TEST-TOPIC-2-0}}
{{ at
org.apache.kafka.streams.processor.internals.StreamTask.findOffsetAndMetadata(StreamTask.java:480)}}
{{ at
org.apache.kafka.streams.processor.internals.StreamTask.committableOffsetsAndMetadata(StreamTask.java:511)}}
{{ at
org.apache.kafka.streams.processor.internals.StreamTask.prepareCommit(StreamTask.java:454)}}
{{ at
org.apache.kafka.streams.processor.internals.TaskExecutor.commitTasksAndMaybeUpdateCommittableOffsets(TaskExecutor.java:145)}}
{{ at
org.apache.kafka.streams.processor.internals.TaskManager.commitTasksAndMaybeUpdateCommittableOffsets(TaskManager.java:2025)}}
{{ at
org.apache.kafka.streams.processor.internals.TaskManager.commit(TaskManager.java:1992)}}
{{ at
org.apache.kafka.streams.processor.internals.StreamThread.maybeCommit(StreamThread.java:1836)}}
{{ at
org.apache.kafka.streams.processor.internals.StreamThread.runOnceWithoutProcessingThreads(StreamThread.java:1288)}}
{{ at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:938)}}
{{ ... 1 more}}
{quote}
was:
{{If there is a KS application that uses a regex source, and we create a new
topic that matches that regex, but produce no messages, the application will
get into an ERROR state.}}
{{if we take
}}{{{}{}}}[RegexSourceIntegrationTest#testRegexRecordsAreProcessedAfterNewTopicCreatedWithMultipleSubtopologies|{{{}https://github.com/apache/kafka/blob/trunk/streams/integration-tests/src/test/java/org/apache/kafka/streams/integration/RegexSourceIntegrationTest.java#L206{}}}]{{{},
but without producing any messages to`{}}}{{{}TEST-TOPIC-2{}}}{{{}` the
problem will reproduce:{}}}
{quote}{{org.apache.kafka.streams.errors.StreamsException:
java.lang.IllegalStateException: Stream task 0_0 does not know the partition:
TEST-TOPIC-2-0}}
{{ at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:981)}}
{{ at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:898)}}
{{Caused by: java.lang.IllegalStateException: Stream task 0_0 does not know the
partition: TEST-TOPIC-2-0}}
{{ at
org.apache.kafka.streams.processor.internals.StreamTask.findOffsetAndMetadata(StreamTask.java:480)}}
{{ at
org.apache.kafka.streams.processor.internals.StreamTask.committableOffsetsAndMetadata(StreamTask.java:511)}}
{{ at
org.apache.kafka.streams.processor.internals.StreamTask.prepareCommit(StreamTask.java:454)}}
{{ at
org.apache.kafka.streams.processor.internals.TaskExecutor.commitTasksAndMaybeUpdateCommittableOffsets(TaskExecutor.java:145)}}
{{ at
org.apache.kafka.streams.processor.internals.TaskManager.commitTasksAndMaybeUpdateCommittableOffsets(TaskManager.java:2025)}}
{{ at
org.apache.kafka.streams.processor.internals.TaskManager.commit(TaskManager.java:1992)}}
{{ at
org.apache.kafka.streams.processor.internals.StreamThread.maybeCommit(StreamThread.java:1836)}}
{{ at
org.apache.kafka.streams.processor.internals.StreamThread.runOnceWithoutProcessingThreads(StreamThread.java:1288)}}
{{ at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:938)}}
{{ ... 1 more}}{quote}
> Error if an empty topic is created when there is a regex source KS
> ------------------------------------------------------------------
>
> Key: KAFKA-19775
> URL: https://issues.apache.org/jira/browse/KAFKA-19775
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 4.0.0, 4.1.0
> Reporter: Nikita Shupletsov
> Assignee: Nikita Shupletsov
> Priority: Major
>
> If there is a KS application that uses a regex source, and we create a new
> topic that matches that regex, but produce no messages, the application will
> get into an {{ERROR}} state.
>
> if we take
> {}[RegexSourceIntegrationTest#testRegexRecordsAreProcessedAfterNewTopicCreatedWithMultipleSubtopologies|#L206{}}}],
> but without producing any messages to {{TEST-TOPIC-2}} the problem will
> reproduce:
> {quote}{{org.apache.kafka.streams.errors.StreamsException:
> java.lang.IllegalStateException: Stream task 0_0 does not know the partition:
> TEST-TOPIC-2-0}}
> {{ at
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:981)}}
> {{ at
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:898)}}
> {{Caused by: java.lang.IllegalStateException: Stream task 0_0 does not know
> the partition: TEST-TOPIC-2-0}}
> {{ at
> org.apache.kafka.streams.processor.internals.StreamTask.findOffsetAndMetadata(StreamTask.java:480)}}
> {{ at
> org.apache.kafka.streams.processor.internals.StreamTask.committableOffsetsAndMetadata(StreamTask.java:511)}}
> {{ at
> org.apache.kafka.streams.processor.internals.StreamTask.prepareCommit(StreamTask.java:454)}}
> {{ at
> org.apache.kafka.streams.processor.internals.TaskExecutor.commitTasksAndMaybeUpdateCommittableOffsets(TaskExecutor.java:145)}}
> {{ at
> org.apache.kafka.streams.processor.internals.TaskManager.commitTasksAndMaybeUpdateCommittableOffsets(TaskManager.java:2025)}}
> {{ at
> org.apache.kafka.streams.processor.internals.TaskManager.commit(TaskManager.java:1992)}}
> {{ at
> org.apache.kafka.streams.processor.internals.StreamThread.maybeCommit(StreamThread.java:1836)}}
> {{ at
> org.apache.kafka.streams.processor.internals.StreamThread.runOnceWithoutProcessingThreads(StreamThread.java:1288)}}
> {{ at
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:938)}}
> {{ ... 1 more}}
> {quote}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
