Seems it does a lot of lstat() during run with option `--check-destination-dir`
Which is fallback in case backup can’t be finished. Hm.
>Среда, 12 мая 2021, 22:44 +09:00 от Andrei Enshin <and.ens...@gmail.com>:
>
>Hi,
>
>Thank you for the explanation.
>
>During backup rdiff-backup did lstat for
>/some/path/rdiff-backup-data/increments/foo/bar
>which returned — ENOENT .
>
>Does it mean it tried to check some file in increments which is not here?
>If it is not in increments, does it mean it was never backed up?
>If all above statements are true, why after backup is done, there is still no
>such file? Is it expected?
>
>
>I’ve just played a bit with rdiff-backup on my local.
>
># at /tmp/tmp.jondxmEQDC
>$ cat > a
>aaa
>^D
>
># at /tmp/tmp.zh49h057dq $ mkdir bckp
>$ rdiff-backup /tmp/tmp.jondxmEQDC bckp/
>$ ls bckp/rdiff-backup-data/increments/
># empty
>
>
>It means after very first backup there is nothing in increments. Let’s add new
>file and do backup once again:
>
># at /tmp/tmp.jondxmEQDC
>$ cat > b
>bbb
>^D
># at /tmp/tmp.zh49h057dq
>$ rdiff-backup /tmp/tmp.jondxmEQDC bckp/
>$ ls bckp/rdiff-backup-data/increments/
>b.2021-05-12T22:11:02+09:00.missing
>
>
>I can see a record for new file with .missing suffix.
>
>However in case of `lstat()` it tries to access something which has not such
>suffix.
>What it tries to access?
>
>
>>Среда, 12 мая 2021, 20:02 +09:00 от Eric L. Zolf < ewl+rdiffbac...@lavar.de >:
>>
>>Hi,
>>
>>first, I don't see anything surprising in what you describe, so all
>>normal AFAICJ.
>>
>>Second, rdiff-backup needs to check each source file/directory and each
>>target, compare them and then copy (or not), so if you have some 2300
>>files to backup, that would sound about right. If the target or the
>>source file doesn't exist, it would give an error.
>>
>>If the files are small or don't have changes, the lstat happen a lot and
>>nothing much else; this is typical random access. It gives a much
>>different access pattern than the copying of bigger files, where more
>>sequential is typically done to read/write the file's data.
>>
>>There is no real way to improve the situation, rdiff-backup goes as fast
>>as it can and I personally don't know an I/O-equivalent of "nice" (and
>>if you limit the I/O, the backup will be even slower).
>>
>>You could try the --no-fsync option to improve speed:
>>
>> --fsync, --no-fsync [opt] do (or not) often sync the file system
>>(_not_ doing it is faster but can be dangerous)
>>
>>And, yes, the `rdiff-backup-data/increments` directory is used by
>>rdiff-backup to keep track of file and directory changes.
>>
>>Hope this helps,
>>Eric
>>
>>On 12/05/2021 07:10, Andrei Enshin via Any discussion of rdiff-backup wrote:
>>>
>>> Hi rdiff-backup folks,
>>>
>>> Since recent, during backing up I can see spike in IOPS up to 500 which
>>> exhaust limit of a VM. Therefore backup process takes very long. I've
>>> straced a bit and what I can see is: many failed lstat() syscalls:
>>> % time seconds usecs/call calls errors syscall
>>> ------ ----------- ----------- --------- --------- ----------------
>>> 42.71 0.040247 9 4608 1420 lstat
>>> 35.41 0.033370 12 2860 getdents
>>> 9.41 0.008865 6 1431 open
>>> 4.63 0.004363 3 1430 close
>>> 4.03 0.003797 3 1431 fstat
>>> 3.75 0.003536 2 1417 getuid
>>> 0.04 0.000039 39 1 unlink
>>> 0.01 0.000013 1 9 read
>>> ------ ----------- ----------- --------- --------- ----------------
>>> 100.00 0.094230 13187 1420 total
>>> Seems rdiff-backup checks existence of some file/dir:
>>> 10:13:16 lstat("/some/path/rdiff-backup-data/increments/foo/bar",
>>> 0x7ffd832fa810) = -1 ENOENT (No such file or directory) <0.000020>
>>> After backup is done, there is still no such file.
>>> Seems the part in path - /rdiff-backup-data/increments/ - is some "config"
>>> for rdiff-backup and probably it tryies to find something but can't?
>>>
>>> What might be wrong in my setup? What would you recommend to check to solve
>>> the issue if it is issue at all?
>>>
>>> ---
>>> Best Regards,
>>> Andrei Enshin
>>>
>
>
>---
>Best Regards,
>Andrei Enshin
>
---
Best Regards,
Andrei Enshin
