YARN currently is unable to handle requests with different resource requirements at the same priority (YARN-314). Using different priorities would likely solve the issue. I believe it would also be safest to avoid using the priority 0 because this is used for the application master.
-Sandy On Mon, Dec 9, 2013 at 12:23 AM, Bill Q <[email protected]> wrote: > No, all default 0. Should I use different priorities? > > Many thanks. > > > Bill > > > On Mon, Dec 9, 2013 at 2:53 AM, Sandy Ryza <[email protected]>wrote: > >> Are you requesting the different container types at different priorities? >> >> -Sandy >> >> >> On Sun, Dec 8, 2013 at 8:52 PM, Bill Q <[email protected]> wrote: >> >>> Hi, >>> I recently implemented an application using YARN. Everything works fine, >>> except for the illogical scheduling behavior of the ResourceManger, which >>> might be caused by the heterogeneity of the cluster. >>> >>> I have two types of computing node in the cluster. 10 nodes with 128G >>> memory and 10 nodes with 64G memory. In the application, I create two types >>> of container. Type A container would ask for 50G of memory and Type B >>> container would ask for 100G of memory. >>> >>> The problem is that if I created 50 Type B containers (100G), and 50 >>> Type A containers (50G), Only Type B container (100G) are running. Instead >>> of running the Type A container (50G), all the nodes with 64G memory are >>> idling. And only the nodes with 128G memory are running Type B container >>> (100G). This doesn't make too much sense to me. >>> >>> If I only created Type A containers (50G), all the nodes would be able >>> run the containers. The 128G node would run two containers simultaneously. >>> So, the problem seems to be connected to the different types of containers. >>> >>> Does anybody have any suggestions to fix this issue? >>> >>> Many thanks. >>> >>> >>> Bill >>> >> >> >
