In the very last output where I specify "--ntasks-per-node=1", I'd expect something lie this:
00: ma02 01: ma03 but yet I get 'hostname' launched on all available cores (16 per node). Thanks. ### command/output ### [software@superblc1:~]$ scontrol show config Configuration data as of 2013-08-02T16:18:53 AccountingStorageBackupHost = (null) AccountingStorageEnforce = associations,limits,qos AccountingStorageHost = sbsrv10 AccountingStorageLoc = N/A AccountingStoragePort = 7031 AccountingStorageType = accounting_storage/slurmdbd AccountingStorageUser = N/A AccountingStoreJobComment = YES AcctGatherEnergyType = acct_gather_energy/none AcctGatherNodeFreq = 0 sec AuthType = auth/munge BackupAddr = (null) BackupController = (null) BatchStartTimeout = 30 sec BOOT_TIME = 2013-08-01T23:38:28 CacheGroups = 0 CheckpointType = checkpoint/none ClusterName = superb CompleteWait = 32 sec ControlAddr = sbsrv10 ControlMachine = sbsrv1,sbsrv9 CryptoType = crypto/munge DebugFlags = (null) DefMemPerCPU = 128 DisableRootJobs = NO EnforcePartLimits = YES Epilog = (null) EpilogMsgTime = 2000 usec EpilogSlurmctld = (null) FastSchedule = 1 FirstJobId = 1 GetEnvTimeout = 2 sec GresTypes = (null) GroupUpdateForce = 1 GroupUpdateTime = 600 sec HASH_VAL = Match HealthCheckInterval = 0 sec HealthCheckProgram = (null) InactiveLimit = 0 sec JobAcctGatherFrequency = 30 sec JobAcctGatherType = jobacct_gather/linux JobCheckpointDir = /var/slurm/checkpoint JobCompHost = localhost JobCompLoc = /var/log/slurm_jobcomp.log JobCompPort = 0 JobCompType = jobcomp/none JobCompUser = root JobCredentialPrivateKey = (null) JobCredentialPublicCertificate = (null) JobFileAppend = 0 JobRequeue = 1 JobSubmitPlugins = (null) KillOnBadExit = 1 KillWait = 30 sec LaunchType = launch/slurm Licenses = (null) LicensesUsed = (null) MailProg = /usr/bin/mail MaxJobCount = 30000 MaxJobId = 1000000000 MaxMemPerCPU = 0 MaxStepCount = 40000 MaxTasksPerNode = 128 MessageTimeout = 10 sec MinJobAge = 300 sec MpiDefault = none MpiParams = ports=12000-12999 NEXT_JOB_ID = 15533 OverTimeLimit = 0 min PluginDir = /usr/local/lib/slurm PlugStackConfig = /etc/slurm PreemptMode = CANCEL PreemptType = preempt/qos PriorityDecayHalfLife = 3-00:00:00 PriorityCalcPeriod = 00:05:00 PriorityFavorSmall = 1 PriorityFlags = 0 PriorityMaxAge = 1-00:00:00 PriorityUsageResetPeriod = NONE PriorityType = priority/multifactor2 PriorityWeightAge = 2000 PriorityWeightFairShare = 6000 PriorityWeightJobSize = 0 PriorityWeightPartition = 0 PriorityWeightQOS = 18000 PrivateData = none ProctrackType = proctrack/cgroup Prolog = (null) PrologSlurmctld = (null) PropagatePrioProcess = 0 PropagateResourceLimits = (null) PropagateResourceLimitsExcept = MEMLOCK RebootProgram = (null) ReconfigFlags = (null) ResumeProgram = (null) ResumeRate = 300 nodes/min ResumeTimeout = 60 sec ResvOverRun = 0 min ReturnToService = 1 SallocDefaultCommand = $SHELL SchedulerParameters = max_job_bf=50,bf_window=1440,default_queue_depth=100 SchedulerPort = 7321 SchedulerRootFilter = 1 SchedulerTimeSlice = 30 sec SchedulerType = sched/backfill SelectType = select/cons_res SelectTypeParameters = CR_CORE_MEMORY SlurmUser = slurm(64030) SlurmctldDebug = info SlurmctldLogFile = /var/log/slurm/slurmctld.log SlurmSchedLogFile = (null) SlurmctldPort = 6817 SlurmctldTimeout = 120 sec SlurmdDebug = info SlurmdLogFile = /var/log/slurm/slurmd.log SlurmdPidFile = /var/run/slurm/slurmd.pid SlurmdPort = 6818 SlurmdSpoolDir = /var/run/slurm/slurmd SlurmdTimeout = 300 sec SlurmdUser = root(0) SlurmSchedLogLevel = 0 SlurmctldPidFile = /var/run/slurm/slurmctld.pid SLURM_CONF = /etc/slurm/slurm.conf SLURM_VERSION = 2.5.7 SrunEpilog = (null) SrunProlog = (null) StateSaveLocation = /var/run/slurm/slurmctld SuspendExcNodes = (null) SuspendExcParts = (null) SuspendProgram = (null) SuspendRate = 60 nodes/min SuspendTime = NONE SuspendTimeout = 30 sec SwitchType = switch/none TaskEpilog = (null) TaskPlugin = task/cgroup TaskPluginParam = (null type) TaskProlog = (null) TmpFS = /scratch TopologyPlugin = topology/none TrackWCKey = 0 TreeWidth = 15 UsePam = 1 UnkillableStepProgram = (null) UnkillableStepTimeout = 60 sec VSizeFactor = 0 percent WaitTime = 0 sec Slurmctld(primary/backup) at sbsrv1,sbsrv9/(NULL) are UP/DOWN [software@superblc1:~]$ salloc -N2 -n32 salloc: Granted job allocation 15530 [software@superblc1:~]$ env | grep SLURM SLURM_NODELIST=ma[02-03] SLURM_NODE_ALIASES=(null) SLURM_MEM_PER_CPU=128 SLURM_NNODES=2 SLURM_JOBID=15530 SLURM_NTASKS=32 SLURM_TASKS_PER_NODE=16(x2) SLURM_JOB_ID=15530 SLURM_SUBMIT_DIR=/srv/share/sbsrv9/home/software SLURM_NPROCS=32 SLURM_JOB_NODELIST=ma[02-03] SLURM_JOB_CPUS_PER_NODE=16(x2) SLURM_JOB_NUM_NODES=2 [software@superblc1:~]$ srun -l hostname | sort 00: ma02 01: ma02 02: ma02 03: ma02 04: ma02 05: ma02 06: ma02 07: ma02 08: ma02 09: ma02 10: ma02 11: ma02 12: ma02 13: ma02 14: ma02 15: ma02 16: ma03 17: ma03 18: ma03 19: ma03 20: ma03 21: ma03 22: ma03 23: ma03 24: ma03 25: ma03 26: ma03 27: ma03 28: ma03 29: ma03 30: ma03 31: ma03 [software@superblc1:~]$ srun --ntasks-per-node=1 -l hostname | sort 00: ma02 01: ma02 02: ma02 03: ma02 04: ma02 05: ma02 06: ma02 07: ma02 08: ma02 09: ma02 10: ma02 11: ma02 12: ma02 13: ma02 14: ma02 15: ma02 16: ma03 17: ma03 18: ma03 19: ma03 20: ma03 21: ma03 22: ma03 23: ma03 24: ma03 25: ma03 26: ma03 27: ma03 28: ma03 29: ma03 30: ma03 31: ma03
