Re: [slurm-users] log rotation for slurmctld.

2020-03-16 Thread Bjørn-Helge Mevik
Marcus Wagner  writes:

> by concidence, I have stumbled today over the troubleshooting slides
> from slug 2019.
>
> SchedMD there explicitly tells us to use SIGUSR2 instead of restart /
> reload / reconfig / SIGHUP.

Right, I forgot about that. :) SIGUSR2 will not even do a reconfigure,
just reopend the log file.  Thanks for the reminder!

-- 
Cheers,
Bjørn-Helge Mevik, dr. scient,
Department for Research Computing, University of Oslo


signature.asc
Description: PGP signature


Re: [slurm-users] log rotation for slurmctld.

2020-03-13 Thread Bjørn-Helge Mevik
navin srivastava  writes:

> can i move the log file  to some other location and then restart.reload of
> slurm service will start a new log file.

Yes, restarting it will start a new log file if the old one is moved
away.  However, also reconfig will do, and you can trigger that by
sending the process a HUP signal.  That way you don't have to restart
the daemon.  We have this in our logrotate file:

postrotate
## Using the newer feature of reconfig when getting a SIGHUP.
kill -hup $(ps -C slurmctld h -o pid)
kill -hup $(ps -C slurmdbd h -o pid)
endscript

(That is for both slurmctld.log and slurmdbd.log.)

-- 
Regards,
Bjørn-Helge Mevik, dr. scient,
Department for Research Computing, University of Oslo


signature.asc
Description: PGP signature


[slurm-users] log rotation for slurmctld.

2020-03-13 Thread navin srivastava
Hi,

i wanted to understand how log rotation of slurmctld works.
in my environment i don't have any logrotation for the slurmctld.log and
now the log file size reached to 125GB.

can i move the log file  to some other location and then restart.reload of
slurm service will start a new log file.i think this should work without
any issues.
am i right or it will create any issue.

Also i need to create a log rotate.is the below config works as it is.i
need to do it on production environment so asking to make sure it will work
fine without any issue.

/var/log/slurm/slurmctld.log {
weekly
missingok
notifempty
sharedscripts
create 0600 slurm slurm
rotate 8
compress
postrotate
  /bin/systemctl reload slurmctld.service > /dev/null 2>/dev/null || true
endscript
}



Regards
Navin.