[jira] [Commented] (KAFKA-5724) AbstractPartitionAssignor does not take into consideration that partition number may start from non-zero

[huxihx (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134658#comment-16134658
 ] 

huxihx commented on KAFKA-5724:
---

[~hachikuji] Did you get any time to look into this jira? Two possible ways to 
solve this problem: 1. Strictly honor the rule that partition must start from 
zero; 2. Pass more information than partition count when invoking `assign` , in 
doing so can we correctly do the assignment based on the metadata. What do you 
think?

> AbstractPartitionAssignor does not take into consideration that partition 
> number may start from non-zero
> 
>
> Key: KAFKA-5724
> URL: https://issues.apache.org/jira/browse/KAFKA-5724
> Project: Kafka
>  Issue Type: Bug
>  Components: clients, consumer
>Affects Versions: 0.11.0.0
>Reporter: Allen Wang
>Assignee: huxihx
>
> In AbstractPartitionAssignor.assign(Cluster metadata, Map Subscription> subscriptions), it invokes assign(partitionsPerTopic, 
> subscriptions). It assumes that partition number starts from 0, and it 
> constructs TopicPartition in the range of [0, partitionsPerTopic). 
> This assumption is not correct. The correct way to handle it is to follow the 
> same approach in producer's DefaultPartitioner, where it uses [0, 
> numberOfPartition) as an index to the actual partition.
> There are use cases where partition number may not start from zero. It can 
> happen if users use advanced tooling to manually craft the partition number 
> when creating topics.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-5724) AbstractPartitionAssignor does not take into consideration that partition number may start from non-zero

[Allen Wang (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123631#comment-16123631
 ] 

Allen Wang commented on KAFKA-5724:
---

[~hachikuji] Please note that TopicCommand can take the replica assignment as 
an argument to create topic. So I believe it does not require much advanced 
tooling to get into this situation.

> AbstractPartitionAssignor does not take into consideration that partition 
> number may start from non-zero
> 
>
> Key: KAFKA-5724
> URL: https://issues.apache.org/jira/browse/KAFKA-5724
> Project: Kafka
>  Issue Type: Bug
>  Components: clients, consumer
>Affects Versions: 0.11.0.0
>Reporter: Allen Wang
>Assignee: huxihx
>
> In AbstractPartitionAssignor.assign(Cluster metadata, Map Subscription> subscriptions), it invokes assign(partitionsPerTopic, 
> subscriptions). It assumes that partition number starts from 0, and it 
> constructs TopicPartition in the range of [0, partitionsPerTopic). 
> This assumption is not correct. The correct way to handle it is to follow the 
> same approach in producer's DefaultPartitioner, where it uses [0, 
> numberOfPartition) as an index to the actual partition.
> There are use cases where partition number may not start from zero. It can 
> happen if users use advanced tooling to manually craft the partition number 
> when creating topics.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-5724) AbstractPartitionAssignor does not take into consideration that partition number may start from non-zero

[Jason Gustafson (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123563#comment-16123563
 ] 

Jason Gustafson commented on KAFKA-5724:


[~allenxwang] I fear this is not the only place we make this assumption. Can 
you clarify specifically how an advanced tool creates a topic with a non-zero 
first partition?

> AbstractPartitionAssignor does not take into consideration that partition 
> number may start from non-zero
> 
>
> Key: KAFKA-5724
> URL: https://issues.apache.org/jira/browse/KAFKA-5724
> Project: Kafka
>  Issue Type: Bug
>  Components: clients, consumer
>Affects Versions: 0.11.0.0
>Reporter: Allen Wang
>Assignee: huxihx
>
> In AbstractPartitionAssignor.assign(Cluster metadata, Map Subscription> subscriptions), it invokes assign(partitionsPerTopic, 
> subscriptions). It assumes that partition number starts from 0, and it 
> constructs TopicPartition in the range of [0, partitionsPerTopic). 
> This assumption is not correct. The correct way to handle it is to follow the 
> same approach in producer's DefaultPartitioner, where it uses [0, 
> numberOfPartition) as an index to the actual partition.
> There are use cases where partition number may not start from zero. It can 
> happen if users use advanced tooling to manually craft the partition number 
> when creating topics.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)