[
https://issues.apache.org/jira/browse/SAMZA-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639329#comment-14639329
]
Navina Ramesh commented on SAMZA-717:
-------------------------------------
[~nickpan47] I don't think we are short of use-cases. For example, the
auto-scaling design requires a custom grouper so that it can allocate tasks to
container based on generated JobModel (SAMZA-719 -
[RB|https://reviews.apache.org/r/36471/])
Having said that, I don't think this API has to be exposed to the user. It
should be controlled by the component that deploys the Samza job.
> Expose the TaskNameGrouper API
> ------------------------------
>
> Key: SAMZA-717
> URL: https://issues.apache.org/jira/browse/SAMZA-717
> Project: Samza
> Issue Type: New Feature
> Reporter: Yan Fang
> Assignee: József Márton Jung
> Priority: Minor
> Fix For: 0.10.0
>
> Attachments: SAMZA-717.0.patch, SAMZA-717.1.patch
>
>
> We now are using the
> [GroupByContainerCount|https://github.com/apache/samza/blob/master/samza-core/src/main/scala/org/apache/samza/container/grouper/task/GroupByContainerCount.scala]
> that extends
> [TaskNameGrouper|https://github.com/apache/samza/blob/master/samza-core/src/main/scala/org/apache/samza/container/grouper/task/TaskNameGrouper.scala]
> to assign TaskModels to ContainerModels (equivalent to assign tasks to
> different containers in YARN world).
> I think it also makes sense that we expose the TaskNameGrouper as an API that
> users can use to implement how they want to assign the TaskModels to the
> ContainerModels.
> This is useful when users have knowledge about the throughput of their
> streams because we are sharing the consumers for all the taskIntances in one
> container. One use case is that users want to put (partition-1, partition-3),
> (partition-2, partition-4) instead of (partition-1, partition-2),
> (partition-3, partition-4), which is current strategy. Because partition-1
> and partition-2 both have a lot of messages coming, while partition-3 and
> partition-4 have fewer messages coming. Of course, when users have enough
> containers (same number as the task number) or all the partitions are equally
> divided, this feature is useless.
> What do you guys think?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
