Hello Erik and Dominic. Thanks very much for the suggestions. I'll most likely try the 4 directory rotation suggestion. That was my first idea as well but thought there might be a simpler way.
Dominic - restoring from these is something relatively frequent (2 - 3 times per month) and is often used as an investigative tool to see what the state of things was at a particular time. For that reason, waiting 30+ minutes to restore the snapshot and then another 30 - 40 minutes to restore the dump can be really cumbersome. I guess this would all be a faster if the files were on an SSD but for now we'll have to live with things as they are. Erik - Can you elaborate on rsync? I use rsync all the time for other needs such as keeping a mirror and in the past I've used a technique with hard links to make incremental backups of a directory tree. But I'm not sure how this could be accomplished where a single large sql dump is concerned. Thanks. On Sat, Oct 16, 2021 at 8:29 AM EricZolf <ewl+rdiffbac...@lavar.de> wrote: > Hello again, > > and to add to this, if you're really only making a backup of one single > huge file, you might not need the overhead of rdiff-backup and could live > with the simpler and slicker rsync. > > KR, Eric > > On 16 October 2021 07:26:19 UTC, Dominic Raferd <domi...@timedicer.co.uk> > wrote: > >You could try using rdiff-backup --no-compression (or > >--no-compression-regexp). This will make backups bigger but should speed > >up backups and restores. > > > >The only built-in compression offered by rdiff-backup is gzip, but if > >you run rdiff-backup --no-compression to a file system using > >ZFS-with-LZ4-compression it will be much faster than gzip for backup and > >for restore, and the compression will be almost as good. Patrik has > >written about this and other benefits of using ZFS, just google > >'rdiff-backup zfs advantages'. > > > >Do you often need to restore files? Is there some other change you could > >make that would reduce the need for frequent restores? In my case > >restores might be slow but they are rare and are lifesavers - so I don't > >mind waiting... > > > >BTW, make sure you are using latest'n'greatest rdiff-backup (currently: > >stable 2.0.5)... > > > >On 14/10/2021 00:03, billy noah wrote: > >> Hello, > >> > >> The older snapshots in my backup scheme take very long to restore as > they > >> are database dumps over 15GB and stored on a rotational drive (which is > >> more cost effective as a backup volume). I noticed that this is > partially > >> due to the fact the rdiff-backup needs to apply each increment as a > patch, > >> stepping backwards until the desired state is achieved. > >> > >> Is there a simple way around this? I wouldn't mind taking up the space > >> needed for a full snapshot if there is some argument that can accomplish > >> this, but I'm trying to avoid the need to build some complex snapshot > >> directory rotation in to my script logic. Currently I'm simply doing: > >> > >> mysqldump --single-transaction --lock-tables=false my_database > > >> /mnt/mirror/daily/dump/my_database.sql > >> rdiff-backup --no-eas --no-acls --no-file-statistics > >> /mnt/mirror/daily/dump/ /mnt/mirror/daily/diff/ > >> rm /mnt/mirror/daily/dump/*sql > >> rdiff-backup --force --remove-older-than 10D /mnt/mirror/daily/diff/ > >>> /dev/null 2>&1 > >> We'd like to keep up to 30 days of increments here but the time needed > to > >> restore something that old would be a bit absurd. I'm thinking > something > >> like 4 full backups with 6 increments each. > >> > >> I hope this makes sense. Thanks! > > > >
