Does anyone know of way to get amount of idle gpu per node or for all cluster ?
sinfo -o %G gives the total amount of gres resource for each node. Is there a way to get the idle amount same as you can get for cpu (%C)?
Perhaps if one use lock file like /dev/nvidia# for each gpu you can check their states?
Thanks in advance, Nadav