[ https://issues.apache.org/jira/browse/KAFKA-2652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14969687#comment-14969687 ]
ASF GitHub Bot commented on KAFKA-2652: --------------------------------------- GitHub user ymatsuda opened a pull request: https://github.com/apache/kafka/pull/353 KAFKA-2652: integrate new group protocol into partition grouping @guozhangwang * added ```PartitionGrouper``` (abstract class) * This class is responsible for grouping partitions to form tasks. * Users may implement this class for custom grouping. * added ```DefaultPartitionGrouper``` * our default implementation of ```PartitionGrouper``` * added ```KafkaStreamingPartitionAssignor``` * We always use this as ```PartitionAssignor``` of stream consumers. * Actual grouping is delegated to ```PartitionGrouper```. * ```TopologyBuilder``` * added ```topicGroups()``` * This returns groups of related topics according to the topology * added ```copartitionSources(sourceNodes...)``` * This is used by DSL layer. It asserts the specified source nodes must be copartitioned. * added ```copartitionGroups()``` which returns groups of copartitioned topics * KStream layer * keep track of source nodes to determine copartition sources when steams are joined * source nodes are set to null when partitioning property is not preserved (ex. ```map()```, ```transform()```), and this indicates the stream is no longer joinable You can merge this pull request into a Git repository by running: $ git pull https://github.com/ymatsuda/kafka grouping Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/353.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #353 ---- commit 708718c1be23fad25fa6206f665cbb619c1b5097 Author: Yasuhiro Matsuda <yasuh...@confluent.io> Date: 2015-10-19T19:38:06Z partition grouping commit d2bae046b5509022e2821a2c5eb08853d228e791 Author: Yasuhiro Matsuda <yasuh...@confluent.io> Date: 2015-10-19T20:19:54Z wip commit 86fa8110b23ee1992fbd19daa08c63a4b427448e Author: Yasuhiro Matsuda <yasuh...@confluent.io> Date: 2015-10-20T20:01:37Z long task id commit 4f4f9ac642ebe0eae33a5c8464309106e9239f2e Author: Yasuhiro Matsuda <yasuh...@confluent.io> Date: 2015-10-20T20:03:15Z Merge branch 'trunk' of github.com:apache/kafka into grouping commit e4ecf39b9ab0b0f4c915a4f43cfe771b1de69f7f Author: Yasuhiro Matsuda <yasuh...@confluent.io> Date: 2015-10-21T19:33:05Z joinability commit 37d72a691173a8fe878ac3d99e8973e72f5675c6 Author: Yasuhiro Matsuda <yasuh...@confluent.io> Date: 2015-10-21T19:33:48Z Merge branch 'trunk' of github.com:apache/kafka into grouping commit f68723bab83c3a3f1c15872f4f24bc932df8198f Author: Yasuhiro Matsuda <yasuh...@confluent.io> Date: 2015-10-22T18:21:31Z partition assignor commit 457cf270222139eae89750781d09abaa07120932 Author: Yasuhiro Matsuda <yasuh...@confluent.io> Date: 2015-10-22T18:21:40Z Merge branch 'trunk' of github.com:apache/kafka into grouping commit 13f3ad703960581229d511287f27345c567b5d3e Author: Yasuhiro Matsuda <yasuh...@confluent.io> Date: 2015-10-22T18:34:52Z complete undoing long taskid commit 98f3bcc1896fd159ccbbd37fc65b1d9d6f568bb9 Author: Yasuhiro Matsuda <yasuh...@confluent.io> Date: 2015-10-22T18:45:38Z fix a test ---- > Incorporate the new consumer protocol with partition-group interface > -------------------------------------------------------------------- > > Key: KAFKA-2652 > URL: https://issues.apache.org/jira/browse/KAFKA-2652 > Project: Kafka > Issue Type: Sub-task > Reporter: Guozhang Wang > Assignee: Yasuhiro Matsuda > Fix For: 0.9.0.1 > > > After KAFKA-2464 is checked in, we need to incorporate the new protocol along > with a partition-group interface. > The first step maybe a couple of pre-defined partitioning scheme that can be > chosen by user from some configs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)