Ok thanks Chris. Is there a quick way to examine how much memory the App Masters are consuming? What is the default setting for yarn.am.containter.memory.mb? How was that value determined? Is it an "ample" setting or is it a bare minimum?
sgg On Dec 18, 2013, at 12:42 PM, Chris Riccomini <[email protected]> wrote: > Hey sgg, > > Samza's model is one YARN AppMaster per-Samza job. This means, if you run > two separate jobs, using run-job.sh, you'll end up with two Ams. > > The overhead of the AM is really just memory (it's not CPU or disk > intensive), and this is adjustable using: > > yarn.am.opts > yarn.am.container.memory.mb > > There is no way to run multiple jobs from the same AM. If you are really > concerned about this, you can collapse your Samza job logic into a single > job. You can even have the job talk to itself, if you need to repartition > data (e.g. Have the output also be the input). > > > Cheers, > Chris > > On 12/18/13 9:36 AM, "sgg" <[email protected]> wrote: > >> Each time I run run-job.sh, I seem to be getting a new separate >> SamzaAppMaster. That seems like a lot of overhead. >> >> Is it somehow possible to have multiple samza jobs share the same >> SamzaAppMaster? >> >> sgg >
