Re: [BackupPC-users] BackuPC 4 hang during transfer

[Adam Goryachev]

2017-09-22 17:24 GMT+02:00 Gandalf Corvotempesta
:

2017-09-22 17:20 GMT+02:00 Les Mikesell :

How does your overall CPU and RAM use look while this is happening?
Remember, your situation is unusual in that you are competing with the
ZFS compression activity.

CPU almost idle, RAM used at 70%, due to ZFS ARC cache (50% of ram)
No swap.


On 23/9/17 02:50, Gandalf Corvotempesta wrote:

Just removed "--checksum" from the BackupPC arguments.
Now is... FAAAST

What i've backupped in about 40hours, now took 60 minutes.

YES: 40 hours => 60 minutes.

Is --checksum really needed ? (checksum is also missing from rsnapshot
arguments, that's why rsnapshot is rocket fast)
I can't be sure for BPC4, but maybe you need more than one full backup 
to get the checkum information available to BPC. I think on v3 you 
needed two full backups before this would happen.
The other point to consider, is that this shows you *DID* have a 
performance issue, but you didn't seem to find it. checksum will 
increase the read load and CPU load on at least the client (and possibly 
BPC server depending on where it gets the checksum info from). So you 
should have seen that you were being limited by disk IO or CPU on either 
BPC server, or the client. I'm not sure of the memory requirement for 
the checksum option, but this too might have been an issue, especially 
if BPC tried to uncompress the file into memory. Also, all of this would 
trash your disk read cache on both systems, further increasing demands 
on the disks.


Whether you need to use --checksum or not, will depend on if you are 
happy to potentially skip backing up some files without knowing about it 
until you need to do a restore. Of course, this is a little contrived, 
as it still requires:

a) size doesn't change
b) timestamp doesn't change
c) content *does* change
That is not a normal process, but it is the corner case that always ends 
up being the most important file ;)




Regards,
Adam


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Installing On Webserver

[R Morgan]

Hi Micheal,

I tested this on a VM and found the the typical apt-get install  does not
interfere with ISPConfig 3.1 or other services on the machine. I have
completed installing BackupPC on my server and I am now in the process of
configuring it and testing.

Thanks for the info.

Randall

On Fri, Sep 22, 2017 at 1:36 PM, Michael Stowe <
michael.st...@member.mensa.org> wrote:

> On 2017-09-21 13:03, R Morgan wrote:
>
> Hi All,
>
> I am looking into installing BackupPC on a Debian 8 webserver I have
> hosted at A2 Hosting. I have successfully installed ISPConfig3.1 and it's
> supporting software and migrated serveral sites to the server.
>
> My concern is that, since ISPConfig is installed and configured on the
> system that the installation may reconfigure Apache2 in a method that my be
> incompatible
>
> with ISPConfig3.1.
>
> Has anyone had any expereince with installing BackupPC on Debian 8 or on a
> system with ISPConfig 3.1?
>
> Any info would be very helpful!
>
> Thanks
>
> R. Morgan
>
> If you install from the tarball, BackupPC doesn't touch any other parts of
> the system and any changes that need to be made (e.g., to Apache) you'll
> need to make yourself.
>
> I can't speak to the Debian package, though it seems unusual to affect
> other package configurations.
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>
>


-- 
If you ask me if it can be done. The answer is YES, it can always be done.
The correct questions however are... What will it cost, and how long will
it take?
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Installing On Webserver

[Michael Stowe]

On 2017-09-21 13:03, R Morgan wrote:

Hi All,

I am looking into installing BackupPC on a Debian 8 webserver I have
hosted at A2 Hosting. I have successfully installed ISPConfig3.1 and
it's supporting software and migrated serveral sites to the server.

My concern is that, since ISPConfig is installed and configured on the
system that the installation may reconfigure Apache2 in a method that
my be incompatible

 with ISPConfig3.1.

Has anyone had any expereince with installing BackupPC on Debian 8 or
on a system with ISPConfig 3.1?

Any info would be very helpful!

Thanks

R. Morgan


If you install from the tarball, BackupPC doesn't touch any other parts 
of the system and any changes that need to be made (e.g., to Apache) 
you'll need to make yourself.


I can't speak to the Debian package, though it seems unusual to affect 
other package configurations.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BackuPC 4 hang during transfer

[Gandalf Corvotempesta]

Just removed "--checksum" from the BackupPC arguments.
Now is... FAAAST

What i've backupped in about 40hours, now took 60 minutes.

YES: 40 hours => 60 minutes.

Is --checksum really needed ? (checksum is also missing from rsnapshot
arguments, that's why rsnapshot is rocket fast)

2017-09-22 17:24 GMT+02:00 Gandalf Corvotempesta
:
> 2017-09-22 17:20 GMT+02:00 Les Mikesell :
>> How does your overall CPU and RAM use look while this is happening?
>> Remember, your situation is unusual in that you are competing with the
>> ZFS compression activity.
>
> CPU almost idle, RAM used at 70%, due to ZFS ARC cache (50% of ram)
> No swap.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BackuPC 4 hang during transfer

[Gandalf Corvotempesta]

2017-09-22 17:20 GMT+02:00 Les Mikesell :
> How does your overall CPU and RAM use look while this is happening?
> Remember, your situation is unusual in that you are competing with the
> ZFS compression activity.

CPU almost idle, RAM used at 70%, due to ZFS ARC cache (50% of ram)
No swap.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BackuPC 4 hang during transfer

[Les Mikesell]

On Fri, Sep 22, 2017 at 9:41 AM, Gandalf Corvotempesta
 wrote:
> Running transfers with loglevel 4, i can see that most of slowdown are
> caused by:
>
> G bpc_file_checksum()
>
> calls.
>

How does your overall CPU and RAM use look while this is happening?
Remember, your situation is unusual in that you are competing with the
ZFS compression activity.

-- 
   Les Mikesell
 lesmikes...@gmail.com

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] BackuPC 4 hang during transfer

[Gandalf Corvotempesta]

Running transfers with loglevel 4, i can see that most of slowdown are
caused by:

G bpc_file_checksum()

calls.

2017-09-22 15:31 GMT+02:00 Gandalf Corvotempesta
:
> Also tried with "nc". I'm able to push/pull 110-120MB/s between these
> two servers.
>
> 2017-09-22 15:22 GMT+02:00 Gandalf Corvotempesta
> :
>> I've made a little bit progress.
>> Slow select calls are always network related but my net is not
>> saturated and I'm able to push 1gbp/s regurally with iperf.
>> Moreover, on client side, rsync is sending files very very fast.
>>
>> Could be a bottleneck in BackupPC network management ?
>>
>> 2017-09-22 15:12 GMT+02:00 Gandalf Corvotempesta
>> :
>>> 2017-09-21 14:37 GMT+02:00 Craig Barratt via BackupPC-users
>>> :
 I recommend running strace -p PID -T on the rsync_bpc process to see what 
 it
 is up to, and how long various system calls take.  I agree your backups
 should run much faster.
>>>
>>> Here is an extract.
>>> Some calls are really slow, more than 1 second.
>>> They seems to be select, right ? Writes seems to be ok, less than 1/10 
>>> second
>>>
>>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
>>> [pid  3452] read(5, "75601860.M846377P28101.xx.gu"..., 8184) = 4081 
>>> <0.35>
>>> [pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
>>> [pid  3453] <... select resumed> )  = 1 (in [3], left {tv_sec=59,
>>> tv_usec=220734}) <0.779284>
>>> [pid  3453] read(3, "\374\17\0\7", 4)   = 4 <0.35>
>>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
>>> [3], left {tv_sec=59, tv_usec=98}) <0.28>
>>> [pid  3453] read(3, ".yy,S=1262266,W=1278715:2,"..., 4092) = 2896 <0.34>
>>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
>>> [3], left {tv_sec=59, tv_usec=98}) <0.29>
>>> [pid  3453] read(3, "383235089.M974673P2358.xx.gu"..., 1196) = 1196 
>>> <0.27>
>>> [pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0}) = 1 (out
>>> [6], left {tv_sec=59, tv_usec=98}) <0.35>
>>> [pid  3453] write(6, "381086351.M234084P17834.xx.g"..., 4087) = 4087 
>>> <0.39>
>>> [pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=59,
>>> tv_usec=220263}) <0.779785>
>>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
>>> [pid  3452] read(5, "381086351.M234084P17834.xx.g"..., 8184) = 4087 
>>> <0.40>
>>> [pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
>>> [pid  3453] <... select resumed> )  = 1 (in [3], left {tv_sec=58,
>>> tv_usec=683238}) <1.316780>
>>> [pid  3453] read(3, "\374\17\0\7", 4)   = 4 <0.56>
>>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
>>> [3], left {tv_sec=59, tv_usec=98}) <0.29>
>>> [pid  3453] read(3,
>>> "t,S=9113,W=9346:2,S\0\231#V\10\300,\270\215\21z\360\250"..., 4092) =
>>> 4092 <0.28>
>>> [pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0}) = 1 (out
>>> [6], left {tv_sec=59, tv_usec=98}) <0.47>
>>> [pid  3453] write(6, "45773421.M45371P9859.xx.gues"..., 4091) = 4091 
>>> <0.51>
>>> [pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=58,
>>> tv_usec=682833}) <1.317230>
>>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
>>> [pid  3452] read(5, "45773421.M45371P9859.xx.gues"..., 8184) = 4091 
>>> <0.33>
>>> [pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
>>> [pid  3453] <... select resumed> )  = 1 (in [3], left {tv_sec=58,
>>> tv_usec=879335}) <1.120688>
>>> [pid  3453] read(3, "\374\17\0\7", 4)   = 4 <0.57>
>>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
>>> [3], left {tv_sec=59, tv_usec=97}) <0.29>
>>> [pid  3453] read(3, ".it,S=17245,W=17685:2,S\0]CR\364\340\327Q\f"...,
>>> 4092) = 4092 <0.26>
>>> [pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0}) = 1 (out
>>> [6], left {tv_sec=59, tv_usec=98}) <0.64>
>>> [pid  3453] write(6, "389990620.M619008P11470.xx.g"..., 4092) = 4092 
>>> <0.51>
>>> [pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=58,
>>> tv_usec=878920}) <1.121130>
>>> [pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0} >> ...>
>>> [pid  3452] read(5,  
>>> [pid  3453] <... select resumed> )  = 1 (out [6], left {tv_sec=59,
>>> tv_usec=98}) <0.29>
>>> [pid  3452] <... read resumed> "389990620.M619008P11470.xx.g"...,
>>> 8184) = 4092 <0.32>
>>> [pid  3453] write(6, "\333\32;^\26\252\321[:>8", 11 
>>> [pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
>>> [pid  3453] <... write resumed> )   = 11 <0.30>
>>> [pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=59,
>>> tv_usec=98}) <0.26>
>>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
>>> [pid  3452] read(5, "\333\32;^\26\252\321[:>8", 8184) = 11 <0.11>
>>> [pid  

Re: [BackupPC-users] BackuPC 4 hang during transfer

[Gandalf Corvotempesta]

Also tried with "nc". I'm able to push/pull 110-120MB/s between these
two servers.

2017-09-22 15:22 GMT+02:00 Gandalf Corvotempesta
:
> I've made a little bit progress.
> Slow select calls are always network related but my net is not
> saturated and I'm able to push 1gbp/s regurally with iperf.
> Moreover, on client side, rsync is sending files very very fast.
>
> Could be a bottleneck in BackupPC network management ?
>
> 2017-09-22 15:12 GMT+02:00 Gandalf Corvotempesta
> :
>> 2017-09-21 14:37 GMT+02:00 Craig Barratt via BackupPC-users
>> :
>>> I recommend running strace -p PID -T on the rsync_bpc process to see what it
>>> is up to, and how long various system calls take.  I agree your backups
>>> should run much faster.
>>
>> Here is an extract.
>> Some calls are really slow, more than 1 second.
>> They seems to be select, right ? Writes seems to be ok, less than 1/10 second
>>
>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
>> [pid  3452] read(5, "75601860.M846377P28101.xx.gu"..., 8184) = 4081 
>> <0.35>
>> [pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
>> [pid  3453] <... select resumed> )  = 1 (in [3], left {tv_sec=59,
>> tv_usec=220734}) <0.779284>
>> [pid  3453] read(3, "\374\17\0\7", 4)   = 4 <0.35>
>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
>> [3], left {tv_sec=59, tv_usec=98}) <0.28>
>> [pid  3453] read(3, ".yy,S=1262266,W=1278715:2,"..., 4092) = 2896 <0.34>
>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
>> [3], left {tv_sec=59, tv_usec=98}) <0.29>
>> [pid  3453] read(3, "383235089.M974673P2358.xx.gu"..., 1196) = 1196 
>> <0.27>
>> [pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0}) = 1 (out
>> [6], left {tv_sec=59, tv_usec=98}) <0.35>
>> [pid  3453] write(6, "381086351.M234084P17834.xx.g"..., 4087) = 4087 
>> <0.39>
>> [pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=59,
>> tv_usec=220263}) <0.779785>
>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
>> [pid  3452] read(5, "381086351.M234084P17834.xx.g"..., 8184) = 4087 
>> <0.40>
>> [pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
>> [pid  3453] <... select resumed> )  = 1 (in [3], left {tv_sec=58,
>> tv_usec=683238}) <1.316780>
>> [pid  3453] read(3, "\374\17\0\7", 4)   = 4 <0.56>
>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
>> [3], left {tv_sec=59, tv_usec=98}) <0.29>
>> [pid  3453] read(3,
>> "t,S=9113,W=9346:2,S\0\231#V\10\300,\270\215\21z\360\250"..., 4092) =
>> 4092 <0.28>
>> [pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0}) = 1 (out
>> [6], left {tv_sec=59, tv_usec=98}) <0.47>
>> [pid  3453] write(6, "45773421.M45371P9859.xx.gues"..., 4091) = 4091 
>> <0.51>
>> [pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=58,
>> tv_usec=682833}) <1.317230>
>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
>> [pid  3452] read(5, "45773421.M45371P9859.xx.gues"..., 8184) = 4091 
>> <0.33>
>> [pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
>> [pid  3453] <... select resumed> )  = 1 (in [3], left {tv_sec=58,
>> tv_usec=879335}) <1.120688>
>> [pid  3453] read(3, "\374\17\0\7", 4)   = 4 <0.57>
>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
>> [3], left {tv_sec=59, tv_usec=97}) <0.29>
>> [pid  3453] read(3, ".it,S=17245,W=17685:2,S\0]CR\364\340\327Q\f"...,
>> 4092) = 4092 <0.26>
>> [pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0}) = 1 (out
>> [6], left {tv_sec=59, tv_usec=98}) <0.64>
>> [pid  3453] write(6, "389990620.M619008P11470.xx.g"..., 4092) = 4092 
>> <0.51>
>> [pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=58,
>> tv_usec=878920}) <1.121130>
>> [pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0} 
>> [pid  3452] read(5,  
>> [pid  3453] <... select resumed> )  = 1 (out [6], left {tv_sec=59,
>> tv_usec=98}) <0.29>
>> [pid  3452] <... read resumed> "389990620.M619008P11470.xx.g"...,
>> 8184) = 4092 <0.32>
>> [pid  3453] write(6, "\333\32;^\26\252\321[:>8", 11 
>> [pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
>> [pid  3453] <... write resumed> )   = 11 <0.30>
>> [pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=59,
>> tv_usec=98}) <0.26>
>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
>> [pid  3452] read(5, "\333\32;^\26\252\321[:>8", 8184) = 11 <0.11>
>> [pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
>> [pid  3453] <... select resumed> )  = 1 (in [3], left {tv_sec=58,
>> tv_usec=696881}) <1.303136>
>> [pid  3453] read(3, "\374\17\0\7", 4)   = 4 <0.28>
>> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
>> [3], left {tv_sec=59, 

Re: [BackupPC-users] BackuPC 4 hang during transfer

[Gandalf Corvotempesta]

I've made a little bit progress.
Slow select calls are always network related but my net is not
saturated and I'm able to push 1gbp/s regurally with iperf.
Moreover, on client side, rsync is sending files very very fast.

Could be a bottleneck in BackupPC network management ?

2017-09-22 15:12 GMT+02:00 Gandalf Corvotempesta
:
> 2017-09-21 14:37 GMT+02:00 Craig Barratt via BackupPC-users
> :
>> I recommend running strace -p PID -T on the rsync_bpc process to see what it
>> is up to, and how long various system calls take.  I agree your backups
>> should run much faster.
>
> Here is an extract.
> Some calls are really slow, more than 1 second.
> They seems to be select, right ? Writes seems to be ok, less than 1/10 second
>
> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
> [pid  3452] read(5, "75601860.M846377P28101.xx.gu"..., 8184) = 4081 <0.35>
> [pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
> [pid  3453] <... select resumed> )  = 1 (in [3], left {tv_sec=59,
> tv_usec=220734}) <0.779284>
> [pid  3453] read(3, "\374\17\0\7", 4)   = 4 <0.35>
> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
> [3], left {tv_sec=59, tv_usec=98}) <0.28>
> [pid  3453] read(3, ".yy,S=1262266,W=1278715:2,"..., 4092) = 2896 <0.34>
> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
> [3], left {tv_sec=59, tv_usec=98}) <0.29>
> [pid  3453] read(3, "383235089.M974673P2358.xx.gu"..., 1196) = 1196 <0.27>
> [pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0}) = 1 (out
> [6], left {tv_sec=59, tv_usec=98}) <0.35>
> [pid  3453] write(6, "381086351.M234084P17834.xx.g"..., 4087) = 4087 
> <0.39>
> [pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=59,
> tv_usec=220263}) <0.779785>
> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
> [pid  3452] read(5, "381086351.M234084P17834.xx.g"..., 8184) = 4087 <0.40>
> [pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
> [pid  3453] <... select resumed> )  = 1 (in [3], left {tv_sec=58,
> tv_usec=683238}) <1.316780>
> [pid  3453] read(3, "\374\17\0\7", 4)   = 4 <0.56>
> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
> [3], left {tv_sec=59, tv_usec=98}) <0.29>
> [pid  3453] read(3,
> "t,S=9113,W=9346:2,S\0\231#V\10\300,\270\215\21z\360\250"..., 4092) =
> 4092 <0.28>
> [pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0}) = 1 (out
> [6], left {tv_sec=59, tv_usec=98}) <0.47>
> [pid  3453] write(6, "45773421.M45371P9859.xx.gues"..., 4091) = 4091 
> <0.51>
> [pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=58,
> tv_usec=682833}) <1.317230>
> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
> [pid  3452] read(5, "45773421.M45371P9859.xx.gues"..., 8184) = 4091 <0.33>
> [pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
> [pid  3453] <... select resumed> )  = 1 (in [3], left {tv_sec=58,
> tv_usec=879335}) <1.120688>
> [pid  3453] read(3, "\374\17\0\7", 4)   = 4 <0.57>
> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
> [3], left {tv_sec=59, tv_usec=97}) <0.29>
> [pid  3453] read(3, ".it,S=17245,W=17685:2,S\0]CR\364\340\327Q\f"...,
> 4092) = 4092 <0.26>
> [pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0}) = 1 (out
> [6], left {tv_sec=59, tv_usec=98}) <0.64>
> [pid  3453] write(6, "389990620.M619008P11470.xx.g"..., 4092) = 4092 
> <0.51>
> [pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=58,
> tv_usec=878920}) <1.121130>
> [pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0} 
> [pid  3452] read(5,  
> [pid  3453] <... select resumed> )  = 1 (out [6], left {tv_sec=59,
> tv_usec=98}) <0.29>
> [pid  3452] <... read resumed> "389990620.M619008P11470.xx.g"...,
> 8184) = 4092 <0.32>
> [pid  3453] write(6, "\333\32;^\26\252\321[:>8", 11 
> [pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
> [pid  3453] <... write resumed> )   = 11 <0.30>
> [pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=59,
> tv_usec=98}) <0.26>
> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
> [pid  3452] read(5, "\333\32;^\26\252\321[:>8", 8184) = 11 <0.11>
> [pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
> [pid  3453] <... select resumed> )  = 1 (in [3], left {tv_sec=58,
> tv_usec=696881}) <1.303136>
> [pid  3453] read(3, "\374\17\0\7", 4)   = 4 <0.28>
> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
> [3], left {tv_sec=59, tv_usec=98}) <0.27>
> [pid  3453] read(3, ".w4w.yy,S=1841,W=1902:2,S\0"..., 4092) = 2896 <0.26>
> [pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
> [3], left {tv_sec=59, tv_usec=999866}) <0.000149>
> [pid  3453] read(3, 

Re: [BackupPC-users] BackuPC 4 hang during transfer

[Gandalf Corvotempesta]

2017-09-21 14:37 GMT+02:00 Craig Barratt via BackupPC-users
:
> I recommend running strace -p PID -T on the rsync_bpc process to see what it
> is up to, and how long various system calls take.  I agree your backups
> should run much faster.

Here is an extract.
Some calls are really slow, more than 1 second.
They seems to be select, right ? Writes seems to be ok, less than 1/10 second

[pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
[pid  3452] read(5, "75601860.M846377P28101.xx.gu"..., 8184) = 4081 <0.35>
[pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
[pid  3453] <... select resumed> )  = 1 (in [3], left {tv_sec=59,
tv_usec=220734}) <0.779284>
[pid  3453] read(3, "\374\17\0\7", 4)   = 4 <0.35>
[pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
[3], left {tv_sec=59, tv_usec=98}) <0.28>
[pid  3453] read(3, ".yy,S=1262266,W=1278715:2,"..., 4092) = 2896 <0.34>
[pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
[3], left {tv_sec=59, tv_usec=98}) <0.29>
[pid  3453] read(3, "383235089.M974673P2358.xx.gu"..., 1196) = 1196 <0.27>
[pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0}) = 1 (out
[6], left {tv_sec=59, tv_usec=98}) <0.35>
[pid  3453] write(6, "381086351.M234084P17834.xx.g"..., 4087) = 4087 <0.39>
[pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=59,
tv_usec=220263}) <0.779785>
[pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
[pid  3452] read(5, "381086351.M234084P17834.xx.g"..., 8184) = 4087 <0.40>
[pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
[pid  3453] <... select resumed> )  = 1 (in [3], left {tv_sec=58,
tv_usec=683238}) <1.316780>
[pid  3453] read(3, "\374\17\0\7", 4)   = 4 <0.56>
[pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
[3], left {tv_sec=59, tv_usec=98}) <0.29>
[pid  3453] read(3,
"t,S=9113,W=9346:2,S\0\231#V\10\300,\270\215\21z\360\250"..., 4092) =
4092 <0.28>
[pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0}) = 1 (out
[6], left {tv_sec=59, tv_usec=98}) <0.47>
[pid  3453] write(6, "45773421.M45371P9859.xx.gues"..., 4091) = 4091 <0.51>
[pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=58,
tv_usec=682833}) <1.317230>
[pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
[pid  3452] read(5, "45773421.M45371P9859.xx.gues"..., 8184) = 4091 <0.33>
[pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
[pid  3453] <... select resumed> )  = 1 (in [3], left {tv_sec=58,
tv_usec=879335}) <1.120688>
[pid  3453] read(3, "\374\17\0\7", 4)   = 4 <0.57>
[pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
[3], left {tv_sec=59, tv_usec=97}) <0.29>
[pid  3453] read(3, ".it,S=17245,W=17685:2,S\0]CR\364\340\327Q\f"...,
4092) = 4092 <0.26>
[pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0}) = 1 (out
[6], left {tv_sec=59, tv_usec=98}) <0.64>
[pid  3453] write(6, "389990620.M619008P11470.xx.g"..., 4092) = 4092 <0.51>
[pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=58,
tv_usec=878920}) <1.121130>
[pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0} 
[pid  3452] read(5,  
[pid  3453] <... select resumed> )  = 1 (out [6], left {tv_sec=59,
tv_usec=98}) <0.29>
[pid  3452] <... read resumed> "389990620.M619008P11470.xx.g"...,
8184) = 4092 <0.32>
[pid  3453] write(6, "\333\32;^\26\252\321[:>8", 11 
[pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
[pid  3453] <... write resumed> )   = 11 <0.30>
[pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=59,
tv_usec=98}) <0.26>
[pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
[pid  3452] read(5, "\333\32;^\26\252\321[:>8", 8184) = 11 <0.11>
[pid  3452] select(6, [5], [], NULL, {tv_sec=60, tv_usec=0} 
[pid  3453] <... select resumed> )  = 1 (in [3], left {tv_sec=58,
tv_usec=696881}) <1.303136>
[pid  3453] read(3, "\374\17\0\7", 4)   = 4 <0.28>
[pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
[3], left {tv_sec=59, tv_usec=98}) <0.27>
[pid  3453] read(3, ".w4w.yy,S=1841,W=1902:2,S\0"..., 4092) = 2896 <0.26>
[pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0}) = 1 (in
[3], left {tv_sec=59, tv_usec=999866}) <0.000149>
[pid  3453] read(3, "s\311\34\344:>:74312186.M702139P19191.x3"...,
1196) = 1196 <0.31>
[pid  3453] select(7, NULL, [6], [6], {tv_sec=60, tv_usec=0}) = 1 (out
[6], left {tv_sec=59, tv_usec=98}) <0.38>
[pid  3453] write(6, "14095904.M963616P21646.xx.gu"..., 4078) = 4078 <0.48>
[pid  3452] <... select resumed> )  = 1 (in [5], left {tv_sec=58,
tv_usec=695988}) <1.304041>
[pid  3453] select(4, [3], [], NULL, {tv_sec=60, tv_usec=0} 
[pid  3452] read(5, "14095904.M963616P21646.xx.gu"..., 8184) = 4078 <0.17>
[pid  3452] select(6, [5], [], NULL, {tv_sec=60,