Thanks a lot. So, in order for the two types of containers mentioned above to work, what priorities should I assign to them? Can you give an example?
Many thanks. Bill On Mon, Dec 9, 2013 at 3:30 AM, Sandy Ryza <[email protected]> wrote: > YARN currently is unable to handle requests with different resource > requirements at the same priority (YARN-314). Using different priorities > would likely solve the issue. I believe it would also be safest to avoid > using the priority 0 because this is used for the application master. > > -Sandy > > > On Mon, Dec 9, 2013 at 12:23 AM, Bill Q <[email protected]> wrote: > >> No, all default 0. Should I use different priorities? >> >> Many thanks. >> >> >> Bill >> >> >> On Mon, Dec 9, 2013 at 2:53 AM, Sandy Ryza <[email protected]>wrote: >> >>> Are you requesting the different container types at different priorities? >>> >>> -Sandy >>> >>> >>> On Sun, Dec 8, 2013 at 8:52 PM, Bill Q <[email protected]> wrote: >>> >>>> Hi, >>>> I recently implemented an application using YARN. Everything works >>>> fine, except for the illogical scheduling behavior of the ResourceManger, >>>> which might be caused by the heterogeneity of the cluster. >>>> >>>> I have two types of computing node in the cluster. 10 nodes with 128G >>>> memory and 10 nodes with 64G memory. In the application, I create two types >>>> of container. Type A container would ask for 50G of memory and Type B >>>> container would ask for 100G of memory. >>>> >>>> The problem is that if I created 50 Type B containers (100G), and 50 >>>> Type A containers (50G), Only Type B container (100G) are running. Instead >>>> of running the Type A container (50G), all the nodes with 64G memory are >>>> idling. And only the nodes with 128G memory are running Type B container >>>> (100G). This doesn't make too much sense to me. >>>> >>>> If I only created Type A containers (50G), all the nodes would be able >>>> run the containers. The 128G node would run two containers simultaneously. >>>> So, the problem seems to be connected to the different types of containers. >>>> >>>> Does anybody have any suggestions to fix this issue? >>>> >>>> Many thanks. >>>> >>>> >>>> Bill >>>> >>> >>> >> >
