Andrew, Our database has roughly 3.4 million rows in the entire schema (including a view, but we also purge job records after 6 months).
After the slurmdbd was upgraded, it took ~9 minutes (manifested by the changes being performed in the log) before the daemon was active again. HTH, John DeSantis 2016-01-13 13:13 GMT-05:00 Trey Dockendorf <[email protected]>: > I didn't time our recent upgrade from 14.03.10 to 15.08.6 but it took > around 30 minutes to an hour for slurmdbd to perform all the database > changes. We have around 10 million jobs in the database. > > I personally setup a test instance of SLURM, including slurmdbd and then > do something like 'mysqldump --opt slurmdbd | mysql slurmdbd_dev' to put my > production database into my test instance and then test the upgrade on the > test cluster. My production and test environments share the same MySQL > server but use different MySQL accounts to access the databases. > > - Trey > > ============================= > > Trey Dockendorf > Systems Analyst I > Texas A&M University > Academy for Advanced Telecommunications and Learning Technologies > Phone: (979)458-2396 > Email: [email protected] > Jabber: [email protected] > > On Tue, Jan 12, 2016 at 4:05 PM, Andrew E. Bruno <[email protected]> > wrote: > >> >> We're planning an upgrade from 14.11.10 to 15.08.6 and in the past the >> slurmdbd upgrades can take a while (~20-30 minutes). Curious if there's >> any way to gauge what we can expect this time? To give an idea, we have >> upwards of ~4.5 million records in our job tables. We frequently get >> these warnings in our logs: >> >> Warning: Note very large processing time from hourly_rollup >> ... >> >> Also, any "best practices" for upgrading the slurmdbd other than: >> >> yum update; systemctl restart slurmdbd >> >> (this usually hangs until the database migrations are complete) >> >> Thanks in advance for any pointers. >> >> Cheers, >> >> --Andrew >> > >
