Hi, I recently implemented an application using YARN. Everything works fine, except for the illogical scheduling behavior of the ResourceManger, which might be caused by the heterogeneity of the cluster.
I have two types of computing node in the cluster. 10 nodes with 128G memory and 10 nodes with 64G memory. In the application, I create two types of container. Type A container would ask for 50G of memory and Type B container would ask for 100G of memory. The problem is that if I created 50 Type B containers (100G), and 50 Type A containers (50G), Only Type B container (100G) are running. Instead of running the Type A container (50G), all the nodes with 64G memory are idling. And only the nodes with 128G memory are running Type B container (100G). This doesn't make too much sense to me. If I only created Type A containers (50G), all the nodes would be able run the containers. The 128G node would run two containers simultaneously. So, the problem seems to be connected to the different types of containers. Does anybody have any suggestions to fix this issue? Many thanks. Bill
