Magnus, As you suggest, CR_Socket and CR_Socket_Memory just prevent more than one job from being allocated CPUs on the same socket. ALLOCATE_FULL_SOCKET=1 actually causes full sockets to be allocated to a job, instead of just individual CPUs within a socket. You can confirm this with "scontrol -d show job". I'm not aware of any other way of doing this. Not sure about the answer to your question regarding memory accounting. Martin Perry Bull Phoenix
From: Magnus Jonsson <[email protected]> To: "slurm-dev" <[email protected]>, Date: 01/09/2013 06:56 AM Subject: [slurm-dev] Accounting of sockets (and memory?) Hello! In our system we have a number of nodes (318) with 4 sockets and 12 cores / socket. The memory bandwidth of the system is tightly related to the socket the core is located in and to ensure that the users get a consistent behaviour between similar jobs we want to allocate whole sockets. We have it configured with: SelectType=select/cons_res SelectTypeParameters=CR_Socket_Memory,CR_CORE_DEFAULT_DIST_BLOCK If a user ask for a "one core job" it will reserve a socket and run the job in that socket with no other jobs running in that socket. Perfect! One small problem is that only one core is accounted for in the slurm accounting system.. not perfect (for us). Is the use of the ALLOCATE_FULL_SOCKET=1 the correct way of altering this behaviour or is there an other way that we are missing? Will the use of ALLOCATE_FULL_SOCKET=1 change the way our users see the system? If they ask for one task will they still just get one? Also on an related matter.. If a user asks for one task but more memory then exist in one socket (for example all the memory available) a user can block the complete node but only one core is accounted for. Is there a way to account for the blocked Socket/Cores or at least acquire the amount of memory requested for each job? I see this information is available in the 'scontrol show job' command but seems to disappear after the job is completed. What options do we have to find the information we are looking for without doing some hack for adding a custom plugins. Best regards, Magnus -- Magnus Jonsson, Developer, HPC2N, Umeå Universitet
