>Memory may be being used by jobs running, or tasks outside the control of
>Slurm running, or possibly NFS buffer cache or similar. You may need to
>start an ssh session on the node and look.
I checked that. For example, on compute-0-1, I see
RealMemory=120705 AllocMem=1024 FreeMem=8442 Sockets=32 Boards=1
I then connected to the node and ran "top"
KiB Mem : 12360248+total, 8623376 free, 5197452 used,
10978166+buff/cache
I then ran
sync; echo 3 > /proc/sys/vm/drop_caches
While "top" shows the caches are deleted
KiB Mem : 12360248+total, 11805062+free, 5202876 used, 348992
buff/cache
Still scontrol shows the same as before
RealMemory=120705 AllocMem=1024 FreeMem=8426 Sockets=32 Boards=1
Regards,
Mahmood