Hello, The nodes are booting diskless(warewulf-3.5-1), and use NFSv3 for everything, beegfs(2015.03.r7-el6) for scratch. I am running slurm-14.11.10 and I have a memory reporting issue:
[tru@visu1 ~]$ sinfo -Nel Tue Dec 15 14:48:48 2015 NODELIST NODES PARTITION STATE CPUS S:C:T MEMORY TMP_DISK WEIGHT FEATURES REASON c[001-040] 40 main* idle 16 2:8:1 1 0 1 (null) none c[041-049,054-056] 12 main* idle 20 2:10:1 1 0 1 (null) none c050 1 main* drained 20 2:10:1 1 0 1 (null) nvidia c[051-053] 3 main* mixed 20 2:10:1 1 0 1 (null) none The MEMORY reports the default amount of memory, instead of the actual memory size: Log on the nodes: [2015-12-15T12:45:00.306] No specialized cores configured by default on this node [2015-12-15T12:45:00.306] Resource spec: Reserved system memory limit not configured for this node [2015-12-15T12:45:00.310] slurmd version 14.11.7 started [2015-12-15T12:45:00.314] slurmd started on Tue, 15 Dec 2015 12:45:00 +0100 [2015-12-15T12:45:00.315] CPUs=16 Boards=1 Sockets=2 Cores=8 Threads=1 Memory=129167 TmpDisk=64583 Uptime=119 CPUSpecList=(null) [tru@c001 ~]$ df -HP Filesystem Size Used Avail Use% Mounted on none 68G 1.2G 67G 2% / tmpfs 68G 0 68G 0% /dev/shm 10.11.3.10:/home 1.1T 865G 235G 79% /baycells/home 10.11.3.10:/scratch 5.5T 867G 4.7T 16% /baycells/scratch 10.11.3.2:/c6/shared 322G 231G 92G 72% /c6/shared 10.11.3.2:/c6/shared/rpm 30G 15G 15G 51% /c6/shared/rpm beegfs_nodev 6.2T 246G 5.9T 4% /mnt/beegfs slurm.conf: ... SelectType=select/cons_res SelectTypeParameters=CR_CPU ... NodeName=c0[01-40] Sockets=2 CoresPerSocket=8 ThreadsPerCore=1 State=UNKNOWN NodeName=c0[41-56] Sockets=2 CoresPerSocket=10 ThreadsPerCore=1 State=UNKNOWN PartitionName=main Nodes=c0[01-56] Default=YES MaxTime=INFINITE State=UP ... Do I have to put RealMemory in the config file? Thanks Tru -- Dr Tru Huynh | http://www.pasteur.fr/research/bis mailto:[email protected] | tel/fax +33 1 45 68 87 37/19 Institut Pasteur, 25-28 rue du Docteur Roux, 75724 Paris CEDEX 15 France
