BTW, I reviewed the StandaloneJob proposal in SAMZA-516 thinking it could have been useful in running Samza in Kubernetes and I was disappointed to find out it was not the case. StandaloneJob goes a step beyond the most basic job implementation as it attempts to handle failures through rebalances managed by the JobCoordinator which is elected via ZoKeeper.
I submit that there is also a need for a simpler type of job, lets call it SimpleJob. This would be a job where, at least until dynamic configuration is implemented, the JobCoordinator is executed once a priori to generate a JobModel, the user can then configure the JobModel in the containers' environment, and execute the containers as he sees fit. Failure handling and container monitoring is then left as something to be handled by whatever system the user chooses to execute the containers. This model can then be used with many of the existing container orchestration systems, whether it be Kubernetes, Docker Swarm, Amazon EC2 Container Service, CoreOS Fleet, Helios, Marathon, Nomad, etc. In fact, KubernetesJob is essentially this SimpleJob proposal, except that it outputs a Kubernetes config file instead of only the JobModel. On Mon, Nov 30, 2015 at 12:31 PM, Jakob Homan <jgho...@gmail.com> wrote: > This is awesome work. Would be interested in opening JIRAs for the > changes you need so we can start to process them?
