[jira] [Commented] (KAFKA-2434) remove roundrobin identical topic constraint in consumer coordinator (old API)
[ https://issues.apache.org/jira/browse/KAFKA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15795525#comment-15795525 ] ASF GitHub Bot commented on KAFKA-2434: --- Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/145 > remove roundrobin identical topic constraint in consumer coordinator (old API) > -- > > Key: KAFKA-2434 > URL: https://issues.apache.org/jira/browse/KAFKA-2434 > Project: Kafka > Issue Type: Sub-task >Reporter: Andrew Olson >Assignee: Andrew Olson > Fix For: 0.10.2.0 > > Attachments: KAFKA-2434.patch > > > The roundrobin strategy algorithm improvement made in KAFKA-2196 should be > applied to the original high-level consumer API as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2434) remove roundrobin identical topic constraint in consumer coordinator (old API)
[ https://issues.apache.org/jira/browse/KAFKA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15167427#comment-15167427 ] Andrew Olson commented on KAFKA-2434: - [~junrao] The pull request for this (https://github.com/apache/kafka/pull/145) has been rebased. Please let me know if there are any concerns that need to be addressed for this patch to be accepted. > remove roundrobin identical topic constraint in consumer coordinator (old API) > -- > > Key: KAFKA-2434 > URL: https://issues.apache.org/jira/browse/KAFKA-2434 > Project: Kafka > Issue Type: Sub-task >Reporter: Andrew Olson >Assignee: Andrew Olson > Attachments: KAFKA-2434.patch > > > The roundrobin strategy algorithm improvement made in KAFKA-2196 should be > applied to the original high-level consumer API as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2434) remove roundrobin identical topic constraint in consumer coordinator (old API)
[ https://issues.apache.org/jira/browse/KAFKA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936850#comment-14936850 ] Andrew Olson commented on KAFKA-2434: - [~junrao] Yes, this was only intended for parity with the original bug-fix implementation in KAFKA-2196. I noticed the distribution uniformity deficiency as well and consequently also submitted a patch for a new "fair" algorithm - see KAFKA-2435. > remove roundrobin identical topic constraint in consumer coordinator (old API) > -- > > Key: KAFKA-2434 > URL: https://issues.apache.org/jira/browse/KAFKA-2434 > Project: Kafka > Issue Type: Sub-task >Reporter: Andrew Olson >Assignee: Andrew Olson > Attachments: KAFKA-2434.patch > > > The roundrobin strategy algorithm improvement made in KAFKA-2196 should be > applied to the original high-level consumer API as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2434) remove roundrobin identical topic constraint in consumer coordinator (old API)
[ https://issues.apache.org/jira/browse/KAFKA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14908997#comment-14908997 ] Jun Rao commented on KAFKA-2434: [~andy_k], thanks for the patch. It seems the algorithm is following: during the round-robin iteration of all the partitions, it just skips over consumers not subscribing to a topic. While this seems to guarantee consistent assignment, it doesn't seem to guarantee uniform distribution. For example, consider the following case. There are 3 consumers c1, c2, and c3. c1 and c2 subscribe to topics t1, t2, and t3, and c3 subscribes to topic t1 only. Each topic has 2 partitions. The algorithm will sort the partitions in the following order: t1-p1, t2-p1, t3-p1, t1-p2, t2-p2, t3-p2, and iterate the consumers in the order of c1, c2, c3. In the end, it seems that t1-p1, t3-p1, t2-p2 will be assigned to c1 and t2-p1, t1-p2, t3-p2 will be assigned to c2, and c3 won't get any partitions. > remove roundrobin identical topic constraint in consumer coordinator (old API) > -- > > Key: KAFKA-2434 > URL: https://issues.apache.org/jira/browse/KAFKA-2434 > Project: Kafka > Issue Type: Sub-task >Reporter: Andrew Olson >Assignee: Andrew Olson > Attachments: KAFKA-2434.patch > > > The roundrobin strategy algorithm improvement made in KAFKA-2196 should be > applied to the original high-level consumer API as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2434) remove roundrobin identical topic constraint in consumer coordinator (old API)
[ https://issues.apache.org/jira/browse/KAFKA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14699668#comment-14699668 ] ASF GitHub Bot commented on KAFKA-2434: --- GitHub user noslowerdna opened a pull request: https://github.com/apache/kafka/pull/145 KAFKA-2434: Remove identical topic subscription constraint for roundrobin strategy in old consumer API You can merge this pull request into a Git repository by running: $ git pull https://github.com/noslowerdna/kafka KAFKA-2434 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/145.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #145 commit d5b93efd9eef20f3f5836d779e6140d5e4a6fc65 Author: Andrew Olson Date: 2015-08-17T15:01:44Z KAFKA-2434: Remove identical topic subscription constraint for roundrobin strategy in old consumer API > remove roundrobin identical topic constraint in consumer coordinator (old API) > -- > > Key: KAFKA-2434 > URL: https://issues.apache.org/jira/browse/KAFKA-2434 > Project: Kafka > Issue Type: Sub-task >Reporter: Andrew Olson >Assignee: Andrew Olson > Attachments: KAFKA-2434.patch > > > The roundrobin strategy algorithm improvement made in KAFKA-2196 should be > applied to the original high-level consumer API as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2434) remove roundrobin identical topic constraint in consumer coordinator (old API)
[ https://issues.apache.org/jira/browse/KAFKA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697264#comment-14697264 ] Andrew Olson commented on KAFKA-2434: - Created reviewboard https://reviews.apache.org/r/37480/diff/ against branch origin/trunk > remove roundrobin identical topic constraint in consumer coordinator (old API) > -- > > Key: KAFKA-2434 > URL: https://issues.apache.org/jira/browse/KAFKA-2434 > Project: Kafka > Issue Type: Sub-task >Reporter: Andrew Olson >Assignee: Andrew Olson > Attachments: KAFKA-2434.patch > > > The roundrobin strategy algorithm improvement made in KAFKA-2196 should be > applied to the original high-level consumer API as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2434) remove roundrobin identical topic constraint in consumer coordinator (old API)
[ https://issues.apache.org/jira/browse/KAFKA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697192#comment-14697192 ] Andrew Olson commented on KAFKA-2434: - I will submit a patch for this change shortly. > remove roundrobin identical topic constraint in consumer coordinator (old API) > -- > > Key: KAFKA-2434 > URL: https://issues.apache.org/jira/browse/KAFKA-2434 > Project: Kafka > Issue Type: Sub-task >Reporter: Andrew Olson >Assignee: Andrew Olson > > The roundrobin strategy algorithm improvement made in KAFKA-2196 should be > applied to the original high-level consumer API as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
