Hi Cliff, On Mon, 2022-02-07 at 11:45 -0800, Mr. Clif wrote: > Hey Eric, > > any ideas on this? How do these diff files normally work? [...]
I'm not an 'rdiff-backup' developer or anything so all you experts out there correct me if I'm wrong but ... IIRC 'rdiff-backup' keeps inode info as part of the metadata for each file. When you mount a filesystem Linux assigns "fake" inode numbers to avoid collisions between filesystems on different devices/partitions/etc.. So if you change the mount point, every file could potentially get a new inode number and, consequently, have changed metadata. That results in 'rdiff-backup' creating a '*.diff*' file for every source file. In addition, since 'rdiff-backup' now thinks the files may have changed it spends a lot of time checking if anything other than metadata has changed which _might_ account for the apparently low throughput. Maybe? Anybody? Cheers Leland -- ------------------------------------------------------------------------------- Leland C. Best | Creationists make it sound as though a 'theory' is lcbpub...@gmail.com | something you dreamt up after being drunk all night. | -- Isaac Asimov PGP Fingerprint | 7B23ACB474299DF0E07F6DEBB8FEE465DCB9B6C3 ------------------------------------------------------------------------------- [...] > > Thanks, > Clif > > On 2/5/22 11:05 AM, Mr. Clif wrote: > > Greetings, > > > > Recently I started experimenting with backing up LVM snapshots of live > > systems instead of the systems themselves. This protects against > > getting complaints about files changing like: > > > > UpdateError: > > 'clif/.mozilla/firefox/5zz41jpw.default/sessionstore-backups/recovery.baklz4' > > Updated mirror temp file > > '/leda.bak/home/clif/.mozilla/firefox/5zz41jpw.default/sessionstore- > > backups/rdiff-backup.tmp.75' > > does not match source > > > > So for example instead of the source being the root dir of a remote > > server, it's now /mnt/vm-backup-snapshot/ on the local server. The > > destination is the same. The first time I tried this I got .diff and > > .diff.gz files for pretty much every file in the filesystem. The vast > > majority of these files are very small like 80-100 bytes. Here is a > > sample: > > > > > > -rwxr-xr-x 1 root root 96 Feb 28 2019 > > tank-rdiff/rdiff-backup-data/increments/bin/dir.2022-02-01T05:20:43- > > 09:00.diff.gz > > -rwxr-xr-x 1 root root 103 Mar 18 2021 > > tank-rdiff/rdiff-backup-data/increments/bin/systemctl.2022-02-01T05:20:43- > > 09:00.diff.gz > > -rwxr-xr-x 1 root root 96 Feb 28 2019 > > tank-rdiff/rdiff-backup-data/increments/bin/touch.2022-02-01T05:20:43- > > 09:00.diff.gz > > -rwxr-xr-x 1 root root 80 Feb 28 2019 > > tank-rdiff/rdiff-backup-data/increments/bin/rm.2022-02-01T05:20:43- > > 09:00.diff.gz > > -rwxr-xr-x 1 root root 93 Jan 21 2019 > > tank-rdiff/rdiff-backup-data/increments/bin/run-parts.2022-02-01T05:20:43- > > 09:00.diff.gz > > -rwxr-xr-x 1 root root 91 Feb 28 2019 > > tank-rdiff/rdiff-backup-data/increments/bin/chown.2022-02-01T05:20:43- > > 09:00.diff.gz > > -rwxr-xr-x 1 root root 94 Mar 18 2021 > > tank-rdiff/rdiff-backup-data/increments/bin/journalctl.2022-02-01T05:20:43- > > 09:00.diff.gz > > -rwxr-xr-x 1 root root 83 Feb 28 2019 > > tank-rdiff/rdiff-backup-data/increments/bin/sync.2022-02-01T05:20:43- > > 09:00.diff.gz > > -rwxr-xr-x 1 root root 96 Mar 18 2021 > > tank-rdiff/rdiff-backup-data/increments/bin/systemd-escape.2022-02- > > 01T05:20:43-09:00.diff.gz > > -rwxr-xr-x 1 root root 109 Mar 18 2021 > > tank-rdiff/rdiff-backup-data/increments/bin/systemd-tty-ask-password- > > agent.2022-02-01T05:20:43-09:00.diff.gz > > -rwxr-xr-x 1 root root 90 Feb 28 2019 > > tank-rdiff/rdiff-backup-data/increments/bin/false.2022-02-01T05:20:43- > > 09:00.diff.gz > > -rwxr-xr-x 1 root root 67 Jan 5 2019 > > tank-rdiff/rdiff-backup-data/increments/bin/zforce.2022-02-01T05:20:43- > > 09:00.diff.gz > > -rwxr-xr-x 1 root root 65 Jan 5 2019 > > tank-rdiff/rdiff-backup-data/increments/bin/znew.2022-02-01T05:20:43- > > 09:00.diff.gz > > -rwxr-xr-x 1 root root 98 Jan 21 2019 > > tank-rdiff/rdiff-backup-data/increments/bin/tempfile.2022-02-01T05:20:43- > > 09:00.diff.gz > > > > The total space used by this increment was around 500MB and the > > largest file was (the only file size that had a suffix): > > > > -rw-r----- 1 root adm 2.1K Jan 22 05:21 > > tank-rdiff/rdiff-backup-data/increments/var/log/auth.log.2022-02-01T05:20:43- > > 09:00.diff.gz > > > > I looked at some of these diff files and they seemed empty except for > > a binary header that I didn't find documentation for in the > > rdiff-backup file format section. I also noticed that occasionally, > > these "empty" diffs show up in normal day to day increments. I realize > > that diffs also can show up when the permissions change but I don't > > think all these files changed. > > > > Is this how it's supposed to work, and could you please explain it to > > me? :-) > > > > Thanks, > > Clif > > > > > >