Hello, I found something in the status output of sinfo to be peculiar.
$ sinfo PARTITION AVAIL TIMELIMIT NODES STATE NODELIST ubique* up infinite 1 down* cal-059 ubique* up infinite 5 drain cal-[172-176] ubique* up infinite 132 mix cal-[005-027,029-032,042-054,056-058,060-074,079-086,089-128,143,145-146,148-149,151-171] ubique* up infinite 37 alloc cal-[001-004,033-041,055,075-078,087-088,129-142,144,147,150] ubique* up infinite 1 down cal-028 Here we can see cal-059, cal-[172-176] and cal-028 are either in state down or drain. So, let's list only the nodes which are down, $ sinfo -t down PARTITION AVAIL TIMELIMIT NODES STATE NODELIST ubique* up infinite 1 down* cal-059 ubique* up infinite 4 drain cal-[173-176] ubique* up infinite 1 down cal-028 All of a sudden, one node disappeared from the list - cal-172. Have I stumbled upon a bug? However, it is shown again if I ask for only the "drain" state, $ sinfo -t drain PARTITION AVAIL TIMELIMIT NODES STATE NODELIST ubique* up infinite 5 drain cal-[172-176] $ sinfo --version slurm 14.11.4 I am unsure if this happens in 14.11.5, but we haven't upgraded to that just yet. If there's anything I can do for verification, please don't hesitate to get in touch. Wbr Andreas ________________________________ Confidentiality Notice: This message is private and may contain confidential and proprietary information. If you have received this message in error, please notify us and remove it from your system and note that you must not copy, distribute or take any action in reliance on it. Any unauthorized use or disclosure of the contents of this message is not permitted and may be unlawful.
