Hi,
Thank you for the explanation.
During backup rdiff-backup did lstat for
/some/path/rdiff-backup-data/increments/foo/bar
which returned — ENOENT .
Does it mean it tried to check some file in increments which is not here?
If it is not in increments, does it mean it was never backed up?
If all above statements are true, why after backup is done, there is still no
such file? Is it expected?
I’ve just played a bit with rdiff-backup on my local.
# at /tmp/tmp.jondxmEQDC
$ cat > a
aaa
^D
# at /tmp/tmp.zh49h057dq $ mkdir bckp
$ rdiff-backup /tmp/tmp.jondxmEQDC bckp/
$ ls bckp/rdiff-backup-data/increments/
# empty
It means after very first backup there is nothing in increments. Let’s add new
file and do backup once again:
# at /tmp/tmp.jondxmEQDC
$ cat > b
bbb
^D
# at /tmp/tmp.zh49h057dq
$ rdiff-backup /tmp/tmp.jondxmEQDC bckp/
$ ls bckp/rdiff-backup-data/increments/
b.2021-05-12T22:11:02+09:00.missing
I can see a record for new file with .missing suffix.
However in case of `lstat()` it tries to access something which has not such
suffix.
What it tries to access?
>Среда, 12 мая 2021, 20:02 +09:00 от Eric L. Zolf <ewl+rdiffbac...@lavar.de>:
>
>Hi,
>
>first, I don't see anything surprising in what you describe, so all
>normal AFAICJ.
>
>Second, rdiff-backup needs to check each source file/directory and each
>target, compare them and then copy (or not), so if you have some 2300
>files to backup, that would sound about right. If the target or the
>source file doesn't exist, it would give an error.
>
>If the files are small or don't have changes, the lstat happen a lot and
>nothing much else; this is typical random access. It gives a much
>different access pattern than the copying of bigger files, where more
>sequential is typically done to read/write the file's data.
>
>There is no real way to improve the situation, rdiff-backup goes as fast
>as it can and I personally don't know an I/O-equivalent of "nice" (and
>if you limit the I/O, the backup will be even slower).
>
>You could try the --no-fsync option to improve speed:
>
> --fsync, --no-fsync [opt] do (or not) often sync the file system
>(_not_ doing it is faster but can be dangerous)
>
>And, yes, the `rdiff-backup-data/increments` directory is used by
>rdiff-backup to keep track of file and directory changes.
>
>Hope this helps,
>Eric
>
>On 12/05/2021 07:10, Andrei Enshin via Any discussion of rdiff-backup wrote:
>>
>> Hi rdiff-backup folks,
>>
>> Since recent, during backing up I can see spike in IOPS up to 500 which
>> exhaust limit of a VM. Therefore backup process takes very long. I've
>> straced a bit and what I can see is: many failed lstat() syscalls:
>> % time seconds usecs/call calls errors syscall
>> ------ ----------- ----------- --------- --------- ----------------
>> 42.71 0.040247 9 4608 1420 lstat
>> 35.41 0.033370 12 2860 getdents
>> 9.41 0.008865 6 1431 open
>> 4.63 0.004363 3 1430 close
>> 4.03 0.003797 3 1431 fstat
>> 3.75 0.003536 2 1417 getuid
>> 0.04 0.000039 39 1 unlink
>> 0.01 0.000013 1 9 read
>> ------ ----------- ----------- --------- --------- ----------------
>> 100.00 0.094230 13187 1420 total
>> Seems rdiff-backup checks existence of some file/dir:
>> 10:13:16 lstat("/some/path/rdiff-backup-data/increments/foo/bar",
>> 0x7ffd832fa810) = -1 ENOENT (No such file or directory) <0.000020>
>> After backup is done, there is still no such file.
>> Seems the part in path - /rdiff-backup-data/increments/ - is some "config"
>> for rdiff-backup and probably it tryies to find something but can't?
>>
>> What might be wrong in my setup? What would you recommend to check to solve
>> the issue if it is issue at all?
>>
>> ---
>> Best Regards,
>> Andrei Enshin
>>
---
Best Regards,
Andrei Enshin
