Re: too many lstat() syscalls, therefore too many IOPS

2021-05-13 Thread Eric L. Zolf
Ups, somehow I sent my e-mail twice... But at least now I know about ionice; never needed it, but it's even installed by default (it's part of util-linux, together with mount, fdisk, etc...). I'm now browsing through the other utilities of this package, full of small jewels. You never stop

Re: too many lstat() syscalls, therefore too many IOPS

2021-05-12 Thread Yves Bellefeuille
"Eric L. Zolf" wrote: > I personally don't know an I/O-equivalent of "nice". Try "ionice". ;-) -- Yves Bellefeuille

Re: too many lstat() syscalls, therefore too many IOPS

2021-05-12 Thread griffin tucker
there is a tool called ionice that comes with the util-linux package, which should help other read/write processes if you are multitasking. ionice shouldn't slow down rdiff-backup if there are no other io processes going on. On Thu, 13 May 2021 at 03:08, Eric L. Zolf wrote: > > Hi, > > first, I

Re: too many lstat() syscalls, therefore too many IOPS

2021-05-12 Thread Eric L. Zolf
Hi, first, I don't see anything surprising in what you describe, so all normal AFAICJ. Second, rdiff-backup needs to check each source file/directory and each target, compare them and then copy (or not), so if you have some 2300 files to backup, that would sound about right. If the target

Re: Re[4]: too many lstat() syscalls, therefore too many IOPS

2021-05-12 Thread EricZolf
We know that check-destination-dir is especially slow, slower than the backup. IIRC there is even an issue open for the regress speed. It just requires time to look into it, and I don't even know if an improvement is possible. The "no such file or directory" thingy doesn't sound normal,

Re[3]: too many lstat() syscalls, therefore too many IOPS

2021-05-12 Thread Andrei Enshin via Any discussion of rdiff-backup
Seems it does a lot of lstat() during run with option `--check-destination-dir` Which is fallback in case backup can’t be finished. Hm.   >Среда, 12 мая 2021, 22:44 +09:00 от Andrei Enshin : >  >Hi, > >Thank you for the explanation. > >During backup rdiff-backup did lstat for

Re[4]: too many lstat() syscalls, therefore too many IOPS

2021-05-12 Thread Andrei Enshin via Any discussion of rdiff-backup
Okay, seems I can see the reason of such behavior. Sorry for disturbing with such questions. We do run backup every 4 hours and seems there is 7200 seconds timeout. It means rdiff-backup will be killed and then we will run it again with `--check-destination-dir` option which causes very

Re[3]: too many lstat() syscalls, therefore too many IOPS

2021-05-12 Thread Andrei Enshin via Any discussion of rdiff-backup
Seems it does a lot of lstat() during run with option `--check-destination-dir` Which is fallback in case backup can’t be finished. Hm.   >Среда, 12 мая 2021, 22:44 +09:00 от Andrei Enshin : >  >Hi, > >Thank you for the explanation. > >During backup rdiff-backup did lstat for

Re[2]: too many lstat() syscalls, therefore too many IOPS

2021-05-12 Thread Andrei Enshin via Any discussion of rdiff-backup
Hi, Thank you for the explanation. During backup rdiff-backup did lstat for /some/path/rdiff-backup-data/increments/foo/bar which returned — ENOENT . Does it mean it tried to check some file in increments which is not here? If it is not in increments, does it mean it was never backed up? If

Re: too many lstat() syscalls, therefore too many IOPS

2021-05-12 Thread Frank Crawford
On Wed, 2021-05-12 at 13:01 +0200, Eric L. Zolf wrote: ... > > There is no real way to improve the situation, rdiff-backup goes as > fast > as it can and I personally don't know an I/O-equivalent of "nice" > (and > if you limit the I/O, the backup will be even slower). The I/O equivalent of

Re: too many lstat() syscalls, therefore too many IOPS

2021-05-12 Thread Eric L. Zolf
Hi, first, I don't see anything surprising in what you describe, so all normal AFAICJ. Second, rdiff-backup needs to check each source file/directory and each target, compare them and then copy (or not), so if you have some 2300 files to backup, that would sound about right. If the target

too many lstat() syscalls, therefore too many IOPS

2021-05-11 Thread Andrei Enshin via Any discussion of rdiff-backup
Hi rdiff-backup folks,   Since recent, during backing up I can see spike in IOPS up to 500 which exhaust limit of a VM. Therefore backup process takes very long. I've straced a bit and what I can see is: many failed lstat() syscalls: % time seconds usecs/call callserrors syscall