This is awesome! Samza will be able to run standalone or on different kinds of clusters, which can serve a wide range of applications. Glad to see this is happening.
Thanks, Xinyu On Tue, Mar 1, 2016 at 8:48 AM, Jagadish Venkatraman <jagadish1...@gmail.com > wrote: > Hi all, > > Currently, the only way to run distributed Samza containers is using Yarn. > There have been requests to run Samza with other resource managers like > Mesos, to run Samza as an embedded library, and to run in Docker containers > managed by ECS/Kubernetes. > > I created SAMZA-881 for this proposing a re-design of the Samza > coordinator. > > > - The new re-design views the JobCoordinator as a component that can > provide the containerId, and JobModel to the SamzaContainer. > - There are pluggable interfaces defined for Leader Election, and > interacting with a Resource Manager like Yarn/Mesos. > - Primitives for dynamically refreshing the job model. > > I've attached a design doc with the Jira ticket, and I'd really appreciate > your feedback on the ticket. > > Thanks, > Jagadish > > -- > Jagadish V, > Graduate Student, > Department of Computer Science, > Stanford University >
