[
https://issues.apache.org/jira/browse/KAFKA-6212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16327518#comment-16327518
]
Guozhang Wang commented on KAFKA-6212:
--------------------------------------
[~ivata89] any news on this issue? Have you figured out the root cause?
> Kafka Streams - Incorrect partition rebalancing
> -----------------------------------------------
>
> Key: KAFKA-6212
> URL: https://issues.apache.org/jira/browse/KAFKA-6212
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.10.0.1
> Environment: Linux
> Reporter: Ivan Atanasov
> Priority: Major
>
> Trying to use streaming with version 0.10.0.1 of kafka but it is not working
> how I'd expect. I realize that this is a fairly old version now but it is
> what we are running and are not in a position to upgrade right now.
> The particular problem I am having is when an extra instance of the streaming
> app is run using the same application ID. What seems to happen is the newly
> introduced instance takes half of the partitions available, which is expected
> but the original instance drops all the partitions it was reading from.
> therefore from then on, data is only read from half the partitions.
> Strangely offsets are still being committed for the other partitions but the
> data from them is not consumed as expected.
> My topology is very simple for now, all it does is a print of the message.
> Also I have tried making both instances use different client IDs and state
> directories.
> Is this a known bug in 0.10.0.1?
> *Logs Below:*
> Instance 1:
> {quote}[2017-11-15 10:41:41,597] INFO [StreamThread-1] Setting newly assigned
> partitions [rawEvents-5, rawEvents-6, rawEvents-3, rawEvents-4, rawEvents-9,
> rawEvents-7, rawEvents-8, rawEvents-1, rawEvents-2, rawEvents-0] for group
> kafka-stream-test
> (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
> [2017-11-15 10:41:41,616] INFO [StreamThread-1] Creating restoration consumer
> client for stream task #0_0
> (org.apache.kafka.streams.processor.internals.StreamTask)
> [2017-11-15 10:41:41,645] INFO [StreamThread-1] Creating restoration consumer
> client for stream task #0_1
> (org.apache.kafka.streams.processor.internals.StreamTask)
> [2017-11-15 10:41:41,645] INFO [StreamThread-1] Creating restoration consumer
> client for stream task #0_2
> (org.apache.kafka.streams.processor.internals.StreamTask)
> [2017-11-15 10:41:41,646] INFO [StreamThread-1] Creating restoration consumer
> client for stream task #0_3
> (org.apache.kafka.streams.processor.internals.StreamTask)
> [2017-11-15 10:41:41,646] INFO [StreamThread-1] Creating restoration consumer
> client for stream task #0_4
> (org.apache.kafka.streams.processor.internals.StreamTask)
> [2017-11-15 10:41:41,646] INFO [StreamThread-1] Creating restoration consumer
> client for stream task #0_5
> (org.apache.kafka.streams.processor.internals.StreamTask)
> [2017-11-15 10:41:41,647] INFO [StreamThread-1] Creating restoration consumer
> client for stream task #0_6
> (org.apache.kafka.streams.processor.internals.StreamTask)
> [2017-11-15 10:41:41,647] INFO [StreamThread-1] Creating restoration consumer
> client for stream task #0_7
> (org.apache.kafka.streams.processor.internals.StreamTask)
> [2017-11-15 10:41:41,647] INFO [StreamThread-1] Creating restoration consumer
> client for stream task #0_8
> (org.apache.kafka.streams.processor.internals.StreamTask)
> [2017-11-15 10:41:41,648] INFO [StreamThread-1] Creating restoration consumer
> client for stream task #0_9
> (org.apache.kafka.streams.processor.internals.StreamTask)
> [2017-11-15 10:42:08,682] INFO [StreamThread-1] Revoking previously assigned
> partitions [rawEvents-5, rawEvents-6, rawEvents-3, rawEvents-4, rawEvents-9,
> rawEvents-7, rawEvents-8, rawEvents-1, rawEvents-2, rawEvents-0] for group
> kafka-stream-test
> (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
> [2017-11-15 10:42:08,682] INFO [StreamThread-1] Removing a task 0_0
> (org.apache.kafka.streams.processor.internals.StreamThread){quote}
> Instance 2:
> {quote}[2017-11-15 10:42:08,827] INFO [StreamThread-1] Successfully joined
> group kafka-stream-test with generation 2
> (org.apache.kafka.clients.consumer.internals.AbstractCoordinator)
> [2017-11-15 10:42:08,829] INFO [StreamThread-1] Setting newly assigned
> partitions [rawEvents-5, rawEvents-3, rawEvents-1, rawEvents-2, rawEvents-0]
> for group kafka-stream-test
> (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
> [2017-11-15 10:42:08,840] INFO [StreamThread-1] Creating restoration consumer
> client for stream task #0_0
> (org.apache.kafka.streams.processor.internals.StreamTask)
> [2017-11-15 10:42:08,869] INFO [StreamThread-1] Creating restoration consumer
> client for stream task #0_1
> (org.apache.kafka.streams.processor.internals.StreamTask)
> [2017-11-15 10:42:08,870] INFO [StreamThread-1] Creating restoration consumer
> client for stream task #0_2
> (org.apache.kafka.streams.processor.internals.StreamTask)
> [2017-11-15 10:42:08,871] INFO [StreamThread-1] Creating restoration consumer
> client for stream task #0_3
> (org.apache.kafka.streams.processor.internals.StreamTask)
> [2017-11-15 10:42:08,872] INFO [StreamThread-1] Creating restoration consumer
> client for stream task #0_5
> (org.apache.kafka.streams.processor.internals.StreamTask){quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
