" densely pack containers on fewer nodes" : quite surprising, +1 with Daemon
You have Yarn labels that can be used for that. Classical example are the need of specific hardware fir some processing. https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeLabel.html Regards, Philippe On Tue, Aug 29, 2017 at 12:53 AM, daemeon reiydelle <daeme...@gmail.com> wrote: > Perhaps you can go into a bit more detail? Especially for e.g. a map job > (or reduce in mapR), this seems like a major antipattern. > > > > > *Daemeon C.M. ReiydelleSan Francisco 1.415.501.0198 > <(415)%20501-0198>London 44 020 8144 9872* > > > On Mon, Aug 28, 2017 at 3:37 PM, Grant Overby <grant.ove...@gmail.com> > wrote: > >> When YARN receives a request for a container, which can be met by many >> nodes, what is the algorithm for determining which node is given the >> container? >> >> Is this tunable? I'd like to densely pack containers on fewer nodes. >> >> A pointer to source code would be nice. >> >> > -- Philippe Kernévez Directeur technique (Suisse), pkerne...@octo.com +41 79 888 33 32 Retrouvez OCTO sur OCTO Talk : http://blog.octo.com OCTO Technology http://www.octo.ch
