[
https://issues.apache.org/jira/browse/SAMZA-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jagadish updated SAMZA-881:
---------------------------
Attachment: SamzaJobCoordinatorRe-designProposal.pdf
> Re-think the Samza Job Coordinator
> ----------------------------------
>
> Key: SAMZA-881
> URL: https://issues.apache.org/jira/browse/SAMZA-881
> Project: Samza
> Issue Type: Improvement
> Reporter: Jagadish
> Assignee: Jagadish
> Attachments: SamzaJobCoordinatorRe-designProposal.pdf
>
>
> Currently, the only way to run Samza containers in distributed mode is using
> Yarn. However, there has been interest to run Samza on top of other resource
> managers with the recent explosion in the # of such systems. Users have also
> requested us to run Samza as a library, and to run Samza on Docker containers
> managed by Kubernetes.
> We must re-think the JobCoordinator functionality as follows:
> 1. ID assignment: Provide an ID to each SamzaContainer.
> 2. JobModel agreement: Ensure containers agree on a JobModel.
> 3. Re-start the SamzaContainer when the job model changes.
> This will arguably require some leader election (depending on how users
> choose to run Samza).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
