I fixed my problem, I have to remove already running jobs , since they took all the memory other jobs will not be able to run. once I removed already running jobs, new jobs just landed where they suppose to run.
Saerda Halifu Overingeniør,VD@IT-avd., UiB > On 17 Jun 2015, at 11:14, Saerda Halifu <[email protected]> wrote: > > Hi, > > I have same problem as you described, and I was only missing DefMemPerCPU, I > have added it , restarted slurmctld , but all pending jobs are still pending, > this should take affect for already pending jobs or ? o > > Thanks! > > Saerda Halifu > Overingeniør,VD@IT-avd., UiB > >> On 09 Jun 2015, at 20:39, Zentz, Scott C. <[email protected] >> <mailto:[email protected]>> wrote: >> >> Yes, that fixed it! For the rest of the list, here are the changes that I >> made to make this work. >> >> Changed “SelectType=select/linear” to “SelectType=select/cons_res” >> Added “SelectTypeParameters=CR_Core_Memory” >> Added “DefMemPerCPU=6000” >> >> Thanks! >> -scz >> >> >> From: Trey Dockendorf [mailto:[email protected] <mailto:[email protected]>] >> Sent: Tuesday, June 09, 2015 1:43 PM >> To: slurm-dev >> Subject: [slurm-dev] Re: Set 1 job per core. >> >> If the goal is to allow jobs to request individual CPUs instead of >> individual nodes, you'll likely want to use select/cons_res. >> http://slurm.schedmd.com/cpu_management.html >> <http://slurm.schedmd.com/cpu_management.html>and >> http://slurm.schedmd.com/cons_res.html >> <http://slurm.schedmd.com/cons_res.html> >> >> - Trey >> >> ============================= >> >> Trey Dockendorf >> Systems Analyst I >> Texas A&M University >> Academy for Advanced Telecommunications and Learning Technologies >> Phone: (979)458-2396 >> Email: [email protected] <mailto:[email protected]> >> Jabber: [email protected] <mailto:[email protected]> >> >> On Tue, Jun 9, 2015 at 12:11 PM, Zentz, Scott C. <[email protected] >> <mailto:[email protected]>> wrote: >> Ok, I tried both DefMemPerNode=6000 and DefMemPerCPU=6000 and neither option >> allowed for more than one job per node. What plugin are most people using >> for their “SelectType”? >> >> Thanks! >> -scz >> >> >> >> >> >> From: Will French [mailto:[email protected] >> <mailto:[email protected]>] >> Sent: Tuesday, June 09, 2015 11:41 AM >> To: slurm-dev >> Subject: [slurm-dev] Re: Set 1 job per core. >> >> >> >> On Jun 9, 2015, at 10:16 AM, Zentz, Scott C. <[email protected] >> <mailto:[email protected]>> wrote: >> >> >> Yes, I have >> >> DefMemPerCPU=6 >> >> Should that be specified in Kb? >> >> No, that’s in MB. I see you’re using SelectType=select/linear. Others with >> experience with this option would probably be more helpful, but you might >> try configuring DefMemPerNode instead of DefMemPerCPU as the SLURM man pages >> state: >> >> DefMemPerNode >> Default real memory size available per allocated node in >> MegaBytes. >> Used to avoid over-subscribing memory and causing paging. >> DefMem- >> PerNode would generally be used if whole nodes are allocated >> to jobs >> (SelectType=select/linear) and resources are shared >> (Shared=yes or >> Shared=force). If not set, the DefMemPerNode value for the >> entire >> cluster will be used. Also see DefMemPerCPU and >> MaxMemPerNode. >> DefMemPerCPU and DefMemPerNode are mutually exclusive. >> NOTE: >> Enforcement of memory limits currently requires enabling of >> account- >> ing, which samples memory use on a periodic basis (data need >> not be >> stored, just collected). >
