On Tuesday, August 14, 2018 10:52:30 PM CEST John Covici wrote: > On Tue, 14 Aug 2018 16:06:21 -0400, > > J. Roeleveld wrote: > > On August 14, 2018 11:42:18 AM UTC, John Covici <cov...@ccs.covici.com> wrote: > > >I use sanoid/syncoid to back up using zfs. Its great, keeps snapshots > > >for as long as I want them (I use 80 days for now). And it keeps > > >hourlies for the last couple of days as well, so I could roll back in > > >case of a problem. Very nice if you use zfs. > > > > I tried sanoid, but it has a few problems which really become annoying > > when you have a lot of datasets: 1) every dataset is handled seperately, > > no use of recursive snapshots when datasets are inside the same tree 2) > > it keeps seperate hourly, daily,.... snapshots, which means it will > > happily create multiple snapshots with only a few seconds difference for > > every dataset around midnight. 3) when rolling back several snapshots, > > there are multiple errors reported because the cache (where does it store > > that?) does not match reality. > > > > Have these been resolved yet? > > > > I ended up writing my own system for this, got some extra intelligence in > > there to work around any possible error condition I have encountered. > Well, I got around your second point by having a special job at 11:59 > pm to create the dailies and the one at midnight works well. I only > do the cron jobs hourly, not every minute like they wanted. > > If your script is not special for you, I would like to see it, maybe I > would use it instead. Things seem to work for now, however with those > modifications.
Current code is too specific for my situation. I think you would be quicker to write something yourself instead of modifying my code. The steps are: - Check if an snapshot needs to be done (incl. type: hourly, daily, weekly, monthly) - If yes: 1) create a recursive snapshot on the entire pool 2) remove unnecessary snapshots (temp, swap, placeholders,not_for_current_type) 3) register snapshots into database 4) clean up old snapshots Because I register actual snapshots and point snapshot-type entries to these, I can quickly determine which snapshots are really unecessary. This also drastically reduces the amount of snapshots on the system. (My SAN currently has 2611 ZFS snapshots). I also found it is far quicker to create a recursive snapshot on the entire pool and then removing all the unecessarily created ones. -- Joost
