[jira] [Updated] (KAFKA-1631) ReplicationFactor and under-replicated partitions incorrect during reassignment
[
https://issues.apache.org/jira/browse/KAFKA-1631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Neha Narkhede updated KAFKA-1631:
-
Assignee: Ewen Cheslack-Postava
ReplicationFactor and under-replicated partitions incorrect during
reassignment
---
Key: KAFKA-1631
URL: https://issues.apache.org/jira/browse/KAFKA-1631
Project: Kafka
Issue Type: Bug
Affects Versions: 0.8.1.1
Reporter: Ryan Berdeen
Assignee: Ewen Cheslack-Postava
Labels: newbie
Attachments: KAFKA-1631-v1.patch
We have a topic with a replication factor of 3. We monitor
UnderReplicatedPartitions as recommended by the documentation.
During a partition reassignment, partitions being reassigned are reported as
under-replicated. Running a describe shows:
{code}
Topic:activity-wal-1PartitionCount:15 ReplicationFactor:5
Configs:
Topic: activity-wal-1 Partition: 0Leader: 14 Replicas:
14,13,12,11,15Isr: 14,12,11,13
Topic: activity-wal-1 Partition: 1Leader: 14 Replicas:
15,14,11 Isr: 14,11
Topic: activity-wal-1 Partition: 2Leader: 11 Replicas:
11,15,12 Isr: 12,11,15
...
{code}
It looks like the displayed replication factor, 5, is simply the number of
replicas listed for the first partition, which includes both brokers in the
current list and those onto which the partition is being reassigned.
Partition 0 is also included in the list when using the
`--under-replicated-partitions` option, even though it is replicated to more
partitions than the true replication factor.
During a reassignment, the under-replicated partitions metric is not usable,
meaning that actual under-replicated partitions can go unnoticed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-1631) ReplicationFactor and under-replicated partitions incorrect during reassignment
[
https://issues.apache.org/jira/browse/KAFKA-1631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ewen Cheslack-Postava updated KAFKA-1631:
-
Attachment: KAFKA-1631-v1.patch
This patch fixes things in a way that I think [~rberdeen] would find sensible.
The issue was that the test for under-replication was comparing the current set
of assigned replicas against the number of in sync replicas. But during the
reassignment that isn't really correct because if you, e.g., move all replicas
to different brokers then you'll have more than the target # of partitions.
The fix is to look up the active set of reassignments and, if one is active for
the partition, use that reassignment info to determine the correct # of
replicas; otherwise, we can fall back on the active set. Note that this does
mean that reassignments that increase the number of replicas will show up as
under-replicated, which I think may be the case [~nehanarkhede] was hoping to
fix. It's arguable which approach is correct (i.e. should the new target # of
replicas apply as soon as the reassignment is issued or once it's completed).
As for the replication factor being reported -- that is the number of currently
assigned replicas for the first partition and has a number of issues.
1. It can be higher than the real target number of replicas as described
above.
2. It's also not really correct to have it on the topic summary line since it
varies by partition.
3. Finally, it's not even just the value for partition 0 because it's just
using the head of a Map.
If we're ok with changing the output formatting here, I can clean that part up
as well, maybe by adding ReplicationFactor to each partition line and making it
use the value used when determining under-replication.
ReplicationFactor and under-replicated partitions incorrect during
reassignment
---
Key: KAFKA-1631
URL: https://issues.apache.org/jira/browse/KAFKA-1631
Project: Kafka
Issue Type: Bug
Affects Versions: 0.8.1.1
Reporter: Ryan Berdeen
Labels: newbie
Attachments: KAFKA-1631-v1.patch
We have a topic with a replication factor of 3. We monitor
UnderReplicatedPartitions as recommended by the documentation.
During a partition reassignment, partitions being reassigned are reported as
under-replicated. Running a describe shows:
{code}
Topic:activity-wal-1PartitionCount:15 ReplicationFactor:5
Configs:
Topic: activity-wal-1 Partition: 0Leader: 14 Replicas:
14,13,12,11,15Isr: 14,12,11,13
Topic: activity-wal-1 Partition: 1Leader: 14 Replicas:
15,14,11 Isr: 14,11
Topic: activity-wal-1 Partition: 2Leader: 11 Replicas:
11,15,12 Isr: 12,11,15
...
{code}
It looks like the displayed replication factor, 5, is simply the number of
replicas listed for the first partition, which includes both brokers in the
current list and those onto which the partition is being reassigned.
Partition 0 is also included in the list when using the
`--under-replicated-partitions` option, even though it is replicated to more
partitions than the true replication factor.
During a reassignment, the under-replicated partitions metric is not usable,
meaning that actual under-replicated partitions can go unnoticed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KAFKA-1631) ReplicationFactor and under-replicated partitions incorrect during reassignment
[
https://issues.apache.org/jira/browse/KAFKA-1631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Neha Narkhede updated KAFKA-1631:
-
Labels: newbie (was: )
ReplicationFactor and under-replicated partitions incorrect during
reassignment
---
Key: KAFKA-1631
URL: https://issues.apache.org/jira/browse/KAFKA-1631
Project: Kafka
Issue Type: Bug
Affects Versions: 0.8.1.1
Reporter: Ryan Berdeen
Labels: newbie
We have a topic with a replication factor of 3. We monitor
UnderReplicatedPartitions as recommended by the documentation.
During a partition reassignment, partitions being reassigned are reported as
under-replicated. Running a describe shows:
{code}
Topic:activity-wal-1PartitionCount:15 ReplicationFactor:5
Configs:
Topic: activity-wal-1 Partition: 0Leader: 14 Replicas:
14,13,12,11,15Isr: 14,12,11,13
Topic: activity-wal-1 Partition: 1Leader: 14 Replicas:
15,14,11 Isr: 14,11
Topic: activity-wal-1 Partition: 2Leader: 11 Replicas:
11,15,12 Isr: 12,11,15
...
{code}
It looks like the displayed replication factor, 5, is simply the number of
replicas listed for the first partition, which includes both brokers in the
current list and those onto which the partition is being reassigned.
Partition 0 is also included in the list when using the
`--under-replicated-partitions` option, even though it is replicated to more
partitions than the true replication factor.
During a reassignment, the under-replicated partitions metric is not usable,
meaning that actual under-replicated partitions can go unnoticed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
