[
https://issues.apache.org/jira/browse/KAFKA-9210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
panpan.liu updated KAFKA-9210:
------------------------------
Description:
kafka broker: 2.0.1
kafka stream client: 2.1.0
# two applications run at the same time
# after some days,I stop one application(in k8s)
# The flollowing log occured and I check the data and find that value is less
than what I expected.
{noformat}
Partitions [flash-app-xmc-worker-share-store-minute-repartition-1]Partitions
[flash-app-xmc-worker-share-store-minute-repartition-1] for changelog
flash-app-xmc-worker-share-store-minute-changelog-12019-11-19 05:50:49.816|WARN
|flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|101|stream-thread
[flash-client-xmc-StreamThread-3] Restoring StreamTasks failed. Deleting
StreamTasks stores to recreate from
scratch.org.apache.kafka.clients.consumer.OffsetOutOfRangeException: Offsets
out of range with no configured reset policy for partitions:
{flash-app-xmc-worker-share-store-minute-changelog-1=6128684} at
org.apache.kafka.clients.consumer.internals.Fetcher.parseCompletedFetch(Fetcher.java:987)2019-11-19
05:50:49.817|INFO
|flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|105|stream-thread
[flash-client-xmc-StreamThread-3] Reinitializing StreamTask TaskId: 10_1
ProcessorTopology: KSTREAM-SOURCE-0000000070: topics:
[flash-app-xmc-worker-share-store-minute-repartition] children:
[KSTREAM-AGGREGATE-0000000067] KSTREAM-AGGREGATE-0000000067: states:
[worker-share-store-minute] children: [KTABLE-TOSTREAM-0000000071]
KTABLE-TOSTREAM-0000000071: children: [KSTREAM-SINK-0000000072]
KSTREAM-SINK-0000000072: topic:
StaticTopicNameExtractor(xmc-worker-share-minute)Partitions
[flash-app-xmc-worker-share-store-minute-repartition-1] for changelog
flash-app-xmc-worker-share-store-minute-changelog-12019-11-19 05:50:49.842|WARN
|flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|101|stream-thread
[flash-client-xmc-StreamThread-3] Restoring StreamTasks failed. Deleting
StreamTasks stores to recreate from
scratch.org.apache.kafka.clients.consumer.OffsetOutOfRangeException: Offsets
out of range with no configured reset policy for partitions:
{flash-app-xmc-worker-share-store-minute-changelog-1=6128684} at
org.apache.kafka.clients.consumer.internals.Fetcher.parseCompletedFetch(Fetcher.java:987)2019-11-19
05:50:49.842|INFO
|flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|105|stream-thread
[flash-client-xmc-StreamThread-3] Reinitializing StreamTask TaskId: 10_1
ProcessorTopology: KSTREAM-SOURCE-0000000070: topics:
[flash-app-xmc-worker-share-store-minute-repartition] children:
[KSTREAM-AGGREGATE-0000000067] KSTREAM-AGGREGATE-0000000067: states:
[worker-share-store-minute] children: [KTABLE-TOSTREAM-0000000071]
KTABLE-TOSTREAM-0000000071: children: [KSTREAM-SINK-0000000072]
KSTREAM-SINK-0000000072: topic:
StaticTopicNameExtractor(xmc-worker-share-minute)Partitions
[flash-app-xmc-worker-share-store-minute-repartition-1] for changelog
flash-app-xmc-worker-share-store-minute-changelog-12019-11-19 05:50:49.905|WARN
|flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|101|stream-thread
[flash-client-xmc-StreamThread-3] Restoring StreamTasks failed. Deleting
StreamTasks stores to recreate from
scratch.org.apache.kafka.clients.consumer.OffsetOutOfRangeException: Offsets
out of range with no configured reset policy for partitions:
{flash-app-xmc-worker-share-store-minute-changelog-1=6128684} at
org.apache.kafka.clients.consumer.internals.Fetcher.parseCompletedFetch(Fetcher.java:987)2019-11-19
05:50:49.906|INFO
|flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|105|stream-thread
[flash-client-xmc-StreamThread-3] Reinitializing StreamTask TaskId: 10_1
ProcessorTopology: KSTREAM-SOURCE-0000000070: topics:
[flash-app-xmc-worker-share-store-minute-repartition] children:
[KSTREAM-AGGREGATE-0000000067] KSTREAM-AGGREGATE-0000000067: states:
[worker-share-store-minute]
{noformat}
was:
```
Partitions [flash-app-xmc-worker-share-store-minute-repartition-1]Partitions
[flash-app-xmc-worker-share-store-minute-repartition-1] for changelog
flash-app-xmc-worker-share-store-minute-changelog-12019-11-19 05:50:49.816|WARN
|flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|101|stream-thread
[flash-client-xmc-StreamThread-3] Restoring StreamTasks failed. Deleting
StreamTasks stores to recreate from
scratch.org.apache.kafka.clients.consumer.OffsetOutOfRangeException: Offsets
out of range with no configured reset policy for partitions:
\{flash-app-xmc-worker-share-store-minute-changelog-1=6128684} at
org.apache.kafka.clients.consumer.internals.Fetcher.parseCompletedFetch(Fetcher.java:987)2019-11-19
05:50:49.817|INFO
|flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|105|stream-thread
[flash-client-xmc-StreamThread-3] Reinitializing StreamTask TaskId: 10_1
ProcessorTopology: KSTREAM-SOURCE-0000000070: topics:
[flash-app-xmc-worker-share-store-minute-repartition] children:
[KSTREAM-AGGREGATE-0000000067] KSTREAM-AGGREGATE-0000000067: states:
[worker-share-store-minute] children: [KTABLE-TOSTREAM-0000000071]
KTABLE-TOSTREAM-0000000071: children: [KSTREAM-SINK-0000000072]
KSTREAM-SINK-0000000072: topic:
StaticTopicNameExtractor(xmc-worker-share-minute)Partitions
[flash-app-xmc-worker-share-store-minute-repartition-1] for changelog
flash-app-xmc-worker-share-store-minute-changelog-12019-11-19 05:50:49.842|WARN
|flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|101|stream-thread
[flash-client-xmc-StreamThread-3] Restoring StreamTasks failed. Deleting
StreamTasks stores to recreate from
scratch.org.apache.kafka.clients.consumer.OffsetOutOfRangeException: Offsets
out of range with no configured reset policy for partitions:
\{flash-app-xmc-worker-share-store-minute-changelog-1=6128684} at
org.apache.kafka.clients.consumer.internals.Fetcher.parseCompletedFetch(Fetcher.java:987)2019-11-19
05:50:49.842|INFO
|flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|105|stream-thread
[flash-client-xmc-StreamThread-3] Reinitializing StreamTask TaskId: 10_1
ProcessorTopology: KSTREAM-SOURCE-0000000070: topics:
[flash-app-xmc-worker-share-store-minute-repartition] children:
[KSTREAM-AGGREGATE-0000000067] KSTREAM-AGGREGATE-0000000067: states:
[worker-share-store-minute] children: [KTABLE-TOSTREAM-0000000071]
KTABLE-TOSTREAM-0000000071: children: [KSTREAM-SINK-0000000072]
KSTREAM-SINK-0000000072: topic:
StaticTopicNameExtractor(xmc-worker-share-minute)Partitions
[flash-app-xmc-worker-share-store-minute-repartition-1] for changelog
flash-app-xmc-worker-share-store-minute-changelog-12019-11-19 05:50:49.905|WARN
|flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|101|stream-thread
[flash-client-xmc-StreamThread-3] Restoring StreamTasks failed. Deleting
StreamTasks stores to recreate from
scratch.org.apache.kafka.clients.consumer.OffsetOutOfRangeException: Offsets
out of range with no configured reset policy for partitions:
\{flash-app-xmc-worker-share-store-minute-changelog-1=6128684} at
org.apache.kafka.clients.consumer.internals.Fetcher.parseCompletedFetch(Fetcher.java:987)2019-11-19
05:50:49.906|INFO
|flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|105|stream-thread
[flash-client-xmc-StreamThread-3] Reinitializing StreamTask TaskId: 10_1
ProcessorTopology: KSTREAM-SOURCE-0000000070: topics:
[flash-app-xmc-worker-share-store-minute-repartition] children:
[KSTREAM-AGGREGATE-0000000067] KSTREAM-AGGREGATE-0000000067: states:
[worker-share-store-minute]
```
> kafka stream loss data
> ----------------------
>
> Key: KAFKA-9210
> URL: https://issues.apache.org/jira/browse/KAFKA-9210
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 2.0.1
> Reporter: panpan.liu
> Priority: Major
>
> kafka broker: 2.0.1
> kafka stream client: 2.1.0
> # two applications run at the same time
> # after some days,I stop one application(in k8s)
> # The flollowing log occured and I check the data and find that value is
> less than what I expected.
>
> {noformat}
> Partitions [flash-app-xmc-worker-share-store-minute-repartition-1]Partitions
> [flash-app-xmc-worker-share-store-minute-repartition-1] for changelog
> flash-app-xmc-worker-share-store-minute-changelog-12019-11-19
> 05:50:49.816|WARN
> |flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|101|stream-thread
> [flash-client-xmc-StreamThread-3] Restoring StreamTasks failed. Deleting
> StreamTasks stores to recreate from
> scratch.org.apache.kafka.clients.consumer.OffsetOutOfRangeException: Offsets
> out of range with no configured reset policy for partitions:
> {flash-app-xmc-worker-share-store-minute-changelog-1=6128684} at
> org.apache.kafka.clients.consumer.internals.Fetcher.parseCompletedFetch(Fetcher.java:987)2019-11-19
> 05:50:49.817|INFO
> |flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|105|stream-thread
> [flash-client-xmc-StreamThread-3] Reinitializing StreamTask TaskId: 10_1
> ProcessorTopology: KSTREAM-SOURCE-0000000070: topics:
> [flash-app-xmc-worker-share-store-minute-repartition] children:
> [KSTREAM-AGGREGATE-0000000067] KSTREAM-AGGREGATE-0000000067: states:
> [worker-share-store-minute] children: [KTABLE-TOSTREAM-0000000071]
> KTABLE-TOSTREAM-0000000071: children: [KSTREAM-SINK-0000000072]
> KSTREAM-SINK-0000000072: topic:
> StaticTopicNameExtractor(xmc-worker-share-minute)Partitions
> [flash-app-xmc-worker-share-store-minute-repartition-1] for changelog
> flash-app-xmc-worker-share-store-minute-changelog-12019-11-19
> 05:50:49.842|WARN
> |flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|101|stream-thread
> [flash-client-xmc-StreamThread-3] Restoring StreamTasks failed. Deleting
> StreamTasks stores to recreate from
> scratch.org.apache.kafka.clients.consumer.OffsetOutOfRangeException: Offsets
> out of range with no configured reset policy for partitions:
> {flash-app-xmc-worker-share-store-minute-changelog-1=6128684} at
> org.apache.kafka.clients.consumer.internals.Fetcher.parseCompletedFetch(Fetcher.java:987)2019-11-19
> 05:50:49.842|INFO
> |flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|105|stream-thread
> [flash-client-xmc-StreamThread-3] Reinitializing StreamTask TaskId: 10_1
> ProcessorTopology: KSTREAM-SOURCE-0000000070: topics:
> [flash-app-xmc-worker-share-store-minute-repartition] children:
> [KSTREAM-AGGREGATE-0000000067] KSTREAM-AGGREGATE-0000000067: states:
> [worker-share-store-minute] children: [KTABLE-TOSTREAM-0000000071]
> KTABLE-TOSTREAM-0000000071: children: [KSTREAM-SINK-0000000072]
> KSTREAM-SINK-0000000072: topic:
> StaticTopicNameExtractor(xmc-worker-share-minute)Partitions
> [flash-app-xmc-worker-share-store-minute-repartition-1] for changelog
> flash-app-xmc-worker-share-store-minute-changelog-12019-11-19
> 05:50:49.905|WARN
> |flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|101|stream-thread
> [flash-client-xmc-StreamThread-3] Restoring StreamTasks failed. Deleting
> StreamTasks stores to recreate from
> scratch.org.apache.kafka.clients.consumer.OffsetOutOfRangeException: Offsets
> out of range with no configured reset policy for partitions:
> {flash-app-xmc-worker-share-store-minute-changelog-1=6128684} at
> org.apache.kafka.clients.consumer.internals.Fetcher.parseCompletedFetch(Fetcher.java:987)2019-11-19
> 05:50:49.906|INFO
> |flash-client-xmc-StreamThread-3|o.a.k.s.p.i.StoreChangelogReader|105|stream-thread
> [flash-client-xmc-StreamThread-3] Reinitializing StreamTask TaskId: 10_1
> ProcessorTopology: KSTREAM-SOURCE-0000000070: topics:
> [flash-app-xmc-worker-share-store-minute-repartition] children:
> [KSTREAM-AGGREGATE-0000000067] KSTREAM-AGGREGATE-0000000067: states:
> [worker-share-store-minute]
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
