It should be possible to do this using placement constraints <https://hadoop.apache.org/docs/r3.1.1/hadoop-yarn/hadoop-yarn-site/PlacementConstraints.html>, though you may need changes in mapreduce to add allocationTags "map" and "reduce" to the containers.
Thanks, Hariharan On Wed, Dec 18, 2019 at 2:04 PM Aakash Sharma <[email protected]> wrote: > Hello Yarn Gurus, > > I am working on a YARN enhancement for optimizing utilization in AWS. > Is it possible to classify a container in resource manager as a Map or > Reduce task? I have looked at the source code of capacity scheduler, and I > dont think it is possible. > If I were to make some placement optimizations in RM to run certain reduce > tasks on a certain type of nodes, without using node labeling, how can I go > about doing this? > > Can anyone please give me some suggestions? > > -- > Thanks, > Aakash >
