[
https://issues.apache.org/jira/browse/SAMZA-881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172449#comment-15172449
]
Jagadish commented on SAMZA-881:
--------------------------------
Please refer
[https://mail-archives.apache.org/mod_mbox/samza-dev/201509.mbox/%3CCAFvExu2Ger6N-bHGwU4fNnzhACyRGjXU--=mltcfgcrwa0s...@mail.gmail.com%3E|
this email thread] for a previous discussion on this topic.
> Re-think the Samza Job Coordinator
> ----------------------------------
>
> Key: SAMZA-881
> URL: https://issues.apache.org/jira/browse/SAMZA-881
> Project: Samza
> Issue Type: Improvement
> Reporter: Jagadish
> Assignee: Jagadish
>
> Currently, the only way to run Samza containers in distributed mode is using
> Yarn. However, there has been interest to run Samza on top of other resource
> managers with the recent explosion in the # of such systems. Users have also
> requested us to run Samza as a library, and to run Samza on Docker containers
> managed by Kubernetes.
> We must re-think the JobCoordinator functionality as follows:
> 1. ID assignment: Provide an ID to each SamzaContainer.
> 2. JobModel agreement: Ensure containers agree on a JobModel.
> 3. Re-start the SamzaContainer when the job model changes.
> This will arguably require some leader election (depending on how users
> choose to run Samza).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
