I have a patch for this that works for a BGQ. I had to use the cpu_cnt in the job record and divide by 16 to get the correct node count. The code I added should continue to work even if the job_ptr->node_cnt is changed in later releases to support sub-midplane reservations. Although at that point it might not be necessary.
To find the correct field and verify that my fix was working correctly I added some debug statements to print various bits from the job_ptr and qos_ptr. When I submit a patch should I leave in the debug statements or remove them? Thanks, Carl ----- Original Message ----- > > I am working in the BGQ simulator to test out new scheduling > configurations and ran into what looks like a bug. > > I created a QOS and tried to limit it to only running on 32 nodes. I > tried setting the GrpNodes, MaxNodes and MaxNodesPerUser. None of > these limits has the expected effect. In each case they apparently > only see one node per job whether the job requests 1, 4 or 32 nodes. > I currently have all 3 of the limits set to 32. I can start up to 32 > jobs of any number of nodes and they will run concurrently. The 33rd > and subsequent jobs will be held with the reason listed as > QOSResourceLimit. I added some debug statements in > slurmctld/acct_policy.c and found that it is indeed adding only one > node per running job to the grp_used_nodes. When looking at the jobs > using scontrol the jobs list the correct number of nodes but > apparently the job_ptr->node_cnt always contains 1. > > I am guessing this has something to with the BGQ being handled as if > each midplane is one node for some things but using a compute node > as a node in other places. > > I am now digging for a better way to count the number of nodes > currently in use by a person or group. Or figuring out how scontrol > shows the correct node count but slurmctld doesn't see it when > scheduling. > > Could this be a symptom of the simulator or do I need to fix this > before moving into production? > > I saw this while working with 2.5.0 and it is the same in 2.5.1. > > Thanks, > Carl > > -- > Carl Schmidtmann > Center for Integrated Research Computing > University of Rochester > -- Carl Schmidtmann Center for Integrated Research Computing University of Rochester
