Hi Martin, Any idea if redhat backported those fixes into rhel 6?
Sent from my iPhone On Apr 25, 2012, at 2:48 PM, [email protected] wrote: > FYI, I think Debian 6 is the 2.6.32 kernel. Memory cgroups are supported in > that version, but we found a significant performance problem using memory > cgroups with jobs that use more than a few cores. This problem is largely > resolved by the 2.6.38 kernel. > Martin > > > > > > > From: Matteo Guglielmi <[email protected]> > To: "slurm-dev" <[email protected]>, > Date: 04/25/2012 11:28 AM > Subject: [slurm-dev] Re: problem with ConstrainRAMSpace and > AllowedRAMSpace > > > > > Hello. > > I'm also very interested in using this RAM > features of cgroups on my debian6 cluster. > > I'm actually running a debian 6 stock kernel, > can I use cgroups for RAM right away or should > I upgrade the kernel to a more recent version? > > > On 04/25/12 16:23, Sefa Arslan wrote: > > > > Hii.. > > > > > > I am trying to use cgroup with slurm. > > > > I have the following lines on my /etc/slurm/cgroup.conf file: > > ### > > # > > # Slurm cgroup support configuration file > > # > > # See man slurm.conf and man cgroup.conf for further > > # information on cgroup configuration parameters > > #-- > > CgroupAutomount=yes > > CgroupReleaseAgentDir="/etc/slurm/cgroup" > > > > ConstrainCores=yes > > TaskAffinity=yes > > ConstrainRAMSpace=yes > > AllowedRAMSpace=10 > > ConstrainSwapSpace=yes > > AllowedSwapSpace=100 > > MaxRAMPercent=10 > > MaxSwapPercent=100 > > > > > > > > > > Our nodes have 128GB ram. I am testing the configuration via xmalloc > > program which takes the given parameter as MegaBytes > > $>salloc -n 1 --nodelist=mercan44 > > $> srun xmalloc 100000 > > > > According to my configuration, I hope that the program will start to use > > swap space when the memory usage reaches 13GB.. (10% of 128GB) > > > > But program continue to use the physical memory even if it reaches 13GB.. > > > > What is the problem with my configuration? > > > > slurm configuration: > > > > scontrol: show config > > Configuration data as of 2012-04-25T17:04:10 > > AccountingStorageBackupHost = (null) > > AccountingStorageEnforce = associations,limits > > AccountingStorageHost = mercan5 > > AccountingStorageLoc = N/A > > AccountingStoragePort = 6819 > > AccountingStorageType = accounting_storage/slurmdbd > > AccountingStorageUser = N/A > > AccountingStoreJobComment = YES > > AuthType = auth/munge > > BackupAddr = (null) > > BackupController = (null) > > BatchStartTimeout = 10 sec > > BOOT_TIME = 2012-04-25T16:33:50 > > CacheGroups = 1 > > CheckpointType = checkpoint/none > > ClusterName = linux > > CompleteWait = 0 sec > > ControlAddr = mercan5 > > ControlMachine = mercan5 > > CryptoType = crypto/munge > > DebugFlags = (null) > > DefMemPerNode = UNLIMITED > > DisableRootJobs = NO > > EnforcePartLimits = YES > > Epilog = (null) > > EpilogMsgTime = 2000 usec > > EpilogSlurmctld = (null) > > FastSchedule = 1 > > FirstJobId = 100000 > > GetEnvTimeout = 2 sec > > GresTypes = (null) > > GroupUpdateForce = 0 > > GroupUpdateTime = 600 sec > > HASH_VAL = Different Ours=0x8997cb77 Slurmctld=0x7742bf02 > > HealthCheckInterval = 0 sec > > HealthCheckProgram = (null) > > InactiveLimit = 0 sec > > JobAcctGatherFrequency = 30 sec > > JobAcctGatherType = jobacct_gather/linux > > JobCheckpointDir = /var/slurm/checkpoint > > JobCompHost = localhost > > JobCompLoc = /var/log/slurm/job_completions > > JobCompPort = 0 > > JobCompType = jobcomp/filetxt > > JobCompUser = root > > JobCredentialPrivateKey = (null) > > JobCredentialPublicCertificate = (null) > > JobFileAppend = 0 > > JobRequeue = 1 > > JobSubmitPlugins = (null) > > KillOnBadExit = 0 > > KillWait = 30 sec > > Licenses = (null) > > MailProg = /bin/mail > > MaxJobCount = 1000000 > > MaxJobId = 4294901760 > > MaxMemPerNode = UNLIMITED > > MaxStepCount = 40000 > > MaxTasksPerNode = 128 > > MessageTimeout = 10 sec > > MinJobAge = 300 sec > > MpiDefault = none > > MpiParams = (null) > > NEXT_JOB_ID = 100678 > > OverTimeLimit = 0 min > > PluginDir = /usr/lib64/slurm > > PlugStackConfig = /etc/slurm/plugstack.conf > > PreemptMode = OFF > > PreemptType = preempt/none > > PriorityDecayHalfLife = 00:00:00 > > PriorityCalcPeriod = 00:05:00 > > PriorityFavorSmall = 0 > > PriorityMaxAge = 1-00:00:00 > > PriorityUsageResetPeriod = NONE > > PriorityType = priority/multifactor > > PriorityWeightAge = 1000 > > PriorityWeightFairShare = 100000 > > PriorityWeightJobSize = 1000 > > PriorityWeightPartition = 10000 > > PriorityWeightQOS = 0 > > PrivateData = none > > ProctrackType = proctrack/cgroup > > Prolog = (null) > > PrologSlurmctld = (null) > > PropagatePrioProcess = 0 > > PropagateResourceLimits = ALL > > PropagateResourceLimitsExcept = (null) > > RebootProgram = (null) > > ReconfigFlags = (null) > > ResumeProgram = (null) > > ResumeRate = 300 nodes/min > > ResumeTimeout = 60 sec > > ResvOverRun = 0 min > > ReturnToService = 0 > > SallocDefaultCommand = (null) > > SchedulerParameters = (null) > > SchedulerPort = 7321 > > SchedulerRootFilter = 1 > > SchedulerTimeSlice = 30 sec > > SchedulerType = sched/backfill > > SelectType = select/cons_res > > SelectTypeParameters = CR_CORE_MEMORY > > SlurmUser = root(0) > > SlurmctldDebug = debug3 > > SlurmctldLogFile = /var/log/slurm/slurmctld.log > > SlurmSchedLogFile = (null) > > SlurmctldPort = 6817 > > SlurmctldTimeout = 300 sec > > SlurmdDebug = debug3 > > SlurmdLogFile = /var/log/slurm/slurmd.log > > SlurmdPidFile = /var/run/slurmd.pid > > SlurmdPort = 6818 > > SlurmdSpoolDir = /tmp/slurmd > > SlurmdTimeout = 300 sec > > SlurmdUser = root(0) > > SlurmSchedLogLevel = 0 > > SlurmctldPidFile = /var/run/slurmctld.pid > > SLURM_CONF = /etc/slurm/slurm.conf > > SLURM_VERSION = 2.4.0-pre4 > > SrunEpilog = (null) > > SrunProlog = (null) > > StateSaveLocation = /tmp > > SuspendExcNodes = (null) > > SuspendExcParts = (null) > > SuspendProgram = (null) > > SuspendRate = 60 nodes/min > > SuspendTime = NONE > > SuspendTimeout = 30 sec > > SwitchType = switch/none > > TaskEpilog = (null) > > TaskPlugin = task/none > > TaskPluginParam = (null type) > > TaskProlog = (null) > > TmpFS = /tmp > > TopologyPlugin = topology/none > > TrackWCKey = 0 > > TreeWidth = 50 > > UsePam = 0 > > UnkillableStepProgram = (null) > > UnkillableStepTimeout = 60 sec > > VSizeFactor = 0 percent > > WaitTime = 0 sec > > > > . > > > > >
