Hi Nicolas, Samza on Yarn (or rather running multiple containers) provides you with
1. *Fault tolerance:* When a machine fails, the YarnJob implementation re-starts your tasks / containers in another machine. 2. *Parallelism:* With multiple containers, you can partition your work among them. If your requirements don't really care about the above, or you have other ways to ensure container liveness using external frameworks (like Kubernetes or Mesos on marathon), then please feel free to use ProcessJobFactory or ThreadJobFactory. Here's a Netflix talk <https://www.youtube.com/watch?v=mWGPlqToT-g&list=PLZDyxA22zzGx34wdHESUux2_V1qfkQ8zx&index=15> in one of our community meetups on how they were running Samza in production without Yarn. We'd also love to have you contribute back your success in running Samza on other environments. Thanks, Jagadish On Wed, Dec 7, 2016 at 2:43 AM, Nicolas Colomer <[email protected]> wrote: > Hello community, > > As mentioned in the documentation > <https://samza.apache.org/learn/documentation/latest/ > jobs/configuration-table.html>, > the *job.factory.class* configuration has 3 possible values: > *ThreadJobFactory*, *ProcessJobFactory*and *YarnJobFactory*. For the first > two, we can read in the description: "This is intended only for > development, not for production deployments.". > > It may be a dumb question but can you elaborate on the reasons of this? > > For instances, we have some Samza jobs that use only one task and are > enough to sustain production traffic. We wonder why we could not run those > jobs in a standalone mode (over Mesos for instance) instead of relying to a > YARN cluster. > > Thanks for your answers. > Nicolas > -- Jagadish V, Graduate Student, Department of Computer Science, Stanford University
