Last night for 8 hours our MySQL server was offline due to a storage failure. As far as I can tell from the slurmctld logs jobs continued to be started and complete successfully. The only errors I saw were in the slurmdbd logs, as expected. What, if any, impact on a SLURM cluster will there be when the accounting database is offline? Do the records that would normally be saved to MySQL get cached while it's offline, or are they forever lost? I found mention that slurmctld will cache data if slurmdbd is offline [1] but unsure if this is true for when the MySQL database is offline.
Thanks, - Trey [1]: http://slurm.schedmd.com/slurm_ug_2012/SUG_Oct2012_DBD.pdf ============================= Trey Dockendorf Systems Analyst I Texas A&M University Academy for Advanced Telecommunications and Learning Technologies Phone: (979)458-2396 Email: [email protected] Jabber: [email protected]
